{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据处理\n",
    "- [缺失值处理](#缺失值处理)\n",
    "\n",
    "## 特征工程\n",
    "- [数值型](#数值型)\n",
    "- [类别型](#类别型)\n",
    "- [时间型](#时间型)\n",
    "- [文本型](#文本型)\n",
    "- [组合特征](#组合特征)\n",
    "\n",
    "## 特征选择\n",
    "- [过滤式](#过滤式)\n",
    "- [包裹式](#包裹式)\n",
    "- [嵌入式](#嵌入式)\n",
    "> 以《泰坦尼克号》救援数据为例，进行特征工程相关的数据练习"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0.数据载入"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<frozen importlib._bootstrap>:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Index(['PassengerId', 'Survived', 'Pclass', 'Name', 'Sex', 'Age', 'SibSp',\n",
       "       'Parch', 'Ticket', 'Fare', 'Cabin', 'Embarked'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "raw_data = pd.read_csv('train.csv')\n",
    "raw_data.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**<font color=red>我们看大概有以下这些字段</font>**<br>\n",
    "PassengerId => 乘客ID<br>\n",
    "Pclass => 乘客等级(1/2/3等舱位)<br>\n",
    "Name => 乘客姓名<br>\n",
    "Sex => 性别<br>\n",
    "Age => 年龄<br>\n",
    "SibSp => 堂兄弟/妹个数<br>\n",
    "Parch => 父母与小孩个数<br>\n",
    "Ticket => 船票信息<br>\n",
    "Fare => 票价<br>\n",
    "Cabin => 客舱<br>\n",
    "Embarked => 登船港口"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "了解数据方式：\n",
    "- head()：对前五行数据进行预览\n",
    "- describe()：展示数据的统计指标：均值、最大最小值等\n",
    "- info()：展示数据总量和非空字段数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>714.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.383838</td>\n",
       "      <td>2.308642</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0.523008</td>\n",
       "      <td>0.381594</td>\n",
       "      <td>32.204208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>257.353842</td>\n",
       "      <td>0.486592</td>\n",
       "      <td>0.836071</td>\n",
       "      <td>14.526497</td>\n",
       "      <td>1.102743</td>\n",
       "      <td>0.806057</td>\n",
       "      <td>49.693429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>223.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>20.125000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.910400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>668.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>31.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>512.329200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId    Survived      Pclass         Age       SibSp  \\\n",
       "count   891.000000  891.000000  891.000000  714.000000  891.000000   \n",
       "mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n",
       "std     257.353842    0.486592    0.836071   14.526497    1.102743   \n",
       "min       1.000000    0.000000    1.000000    0.420000    0.000000   \n",
       "25%     223.500000    0.000000    2.000000   20.125000    0.000000   \n",
       "50%     446.000000    0.000000    3.000000   28.000000    0.000000   \n",
       "75%     668.500000    1.000000    3.000000   38.000000    1.000000   \n",
       "max     891.000000    1.000000    3.000000   80.000000    8.000000   \n",
       "\n",
       "            Parch        Fare  \n",
       "count  891.000000  891.000000  \n",
       "mean     0.381594   32.204208  \n",
       "std      0.806057   49.693429  \n",
       "min      0.000000    0.000000  \n",
       "25%      0.000000    7.910400  \n",
       "50%      0.000000   14.454200  \n",
       "75%      0.000000   31.000000  \n",
       "max      6.000000  512.329200  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw_data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      " #   Column       Non-Null Count  Dtype  \n",
      "---  ------       --------------  -----  \n",
      " 0   PassengerId  891 non-null    int64  \n",
      " 1   Survived     891 non-null    int64  \n",
      " 2   Pclass       891 non-null    int64  \n",
      " 3   Name         891 non-null    object \n",
      " 4   Sex          891 non-null    object \n",
      " 5   Age          714 non-null    float64\n",
      " 6   SibSp        891 non-null    int64  \n",
      " 7   Parch        891 non-null    int64  \n",
      " 8   Ticket       891 non-null    object \n",
      " 9   Fare         891 non-null    float64\n",
      " 10  Cabin        204 non-null    object \n",
      " 11  Embarked     889 non-null    object \n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.7+ KB\n"
     ]
    }
   ],
   "source": [
    "raw_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.数据处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 缺失值处理\n",
    "> 只对缺失值列进行处理，后续需要和原始数据进行合并\n",
    "\n",
    "- pandas fillna\n",
    "- sklearn Imputer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 1-1.使用 pandas的 fillna函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function fillna in module pandas.core.frame:\n",
      "\n",
      "fillna(self, value=None, method=None, axis=None, inplace=False, limit=None, downcast=None) -> Union[ForwardRef('DataFrame'), NoneType]\n",
      "    Fill NA/NaN values using the specified method.\n",
      "    \n",
      "    Parameters\n",
      "    ----------\n",
      "    value : scalar, dict, Series, or DataFrame\n",
      "        Value to use to fill holes (e.g. 0), alternately a\n",
      "        dict/Series/DataFrame of values specifying which value to use for\n",
      "        each index (for a Series) or column (for a DataFrame).  Values not\n",
      "        in the dict/Series/DataFrame will not be filled. This value cannot\n",
      "        be a list.\n",
      "    method : {'backfill', 'bfill', 'pad', 'ffill', None}, default None\n",
      "        Method to use for filling holes in reindexed Series\n",
      "        pad / ffill: propagate last valid observation forward to next valid\n",
      "        backfill / bfill: use next valid observation to fill gap.\n",
      "    axis : {0 or 'index', 1 or 'columns'}\n",
      "        Axis along which to fill missing values.\n",
      "    inplace : bool, default False\n",
      "        If True, fill in-place. Note: this will modify any\n",
      "        other views on this object (e.g., a no-copy slice for a column in a\n",
      "        DataFrame).\n",
      "    limit : int, default None\n",
      "        If method is specified, this is the maximum number of consecutive\n",
      "        NaN values to forward/backward fill. In other words, if there is\n",
      "        a gap with more than this number of consecutive NaNs, it will only\n",
      "        be partially filled. If method is not specified, this is the\n",
      "        maximum number of entries along the entire axis where NaNs will be\n",
      "        filled. Must be greater than 0 if not None.\n",
      "    downcast : dict, default is None\n",
      "        A dict of item->dtype of what to downcast if possible,\n",
      "        or the string 'infer' which will try to downcast to an appropriate\n",
      "        equal type (e.g. float64 to int64 if possible).\n",
      "    \n",
      "    Returns\n",
      "    -------\n",
      "    DataFrame or None\n",
      "        Object with missing values filled or None if ``inplace=True``.\n",
      "    \n",
      "    See Also\n",
      "    --------\n",
      "    interpolate : Fill NaN values using interpolation.\n",
      "    reindex : Conform object to new index.\n",
      "    asfreq : Convert TimeSeries to specified frequency.\n",
      "    \n",
      "    Examples\n",
      "    --------\n",
      "    >>> df = pd.DataFrame([[np.nan, 2, np.nan, 0],\n",
      "    ...                    [3, 4, np.nan, 1],\n",
      "    ...                    [np.nan, np.nan, np.nan, 5],\n",
      "    ...                    [np.nan, 3, np.nan, 4]],\n",
      "    ...                   columns=list('ABCD'))\n",
      "    >>> df\n",
      "         A    B   C  D\n",
      "    0  NaN  2.0 NaN  0\n",
      "    1  3.0  4.0 NaN  1\n",
      "    2  NaN  NaN NaN  5\n",
      "    3  NaN  3.0 NaN  4\n",
      "    \n",
      "    Replace all NaN elements with 0s.\n",
      "    \n",
      "    >>> df.fillna(0)\n",
      "        A   B   C   D\n",
      "    0   0.0 2.0 0.0 0\n",
      "    1   3.0 4.0 0.0 1\n",
      "    2   0.0 0.0 0.0 5\n",
      "    3   0.0 3.0 0.0 4\n",
      "    \n",
      "    We can also propagate non-null values forward or backward.\n",
      "    \n",
      "    >>> df.fillna(method='ffill')\n",
      "        A   B   C   D\n",
      "    0   NaN 2.0 NaN 0\n",
      "    1   3.0 4.0 NaN 1\n",
      "    2   3.0 4.0 NaN 5\n",
      "    3   3.0 3.0 NaN 4\n",
      "    \n",
      "    Replace all NaN elements in column 'A', 'B', 'C', and 'D', with 0, 1,\n",
      "    2, and 3 respectively.\n",
      "    \n",
      "    >>> values = {'A': 0, 'B': 1, 'C': 2, 'D': 3}\n",
      "    >>> df.fillna(value=values)\n",
      "        A   B   C   D\n",
      "    0   0.0 2.0 2.0 0\n",
      "    1   3.0 4.0 2.0 1\n",
      "    2   0.0 1.0 2.0 5\n",
      "    3   0.0 3.0 2.0 4\n",
      "    \n",
      "    Only replace the first NaN element.\n",
      "    \n",
      "    >>> df.fillna(value=values, limit=1)\n",
      "        A   B   C   D\n",
      "    0   0.0 2.0 2.0 0\n",
      "    1   3.0 4.0 NaN 1\n",
      "    2   NaN 1.0 NaN 5\n",
      "    3   NaN 3.0 NaN 4\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 查询 fillna 函数的帮助文档\n",
    "help(pd.DataFrame.fillna)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      22.0\n",
       "1      38.0\n",
       "2      26.0\n",
       "3      35.0\n",
       "4      35.0\n",
       "       ... \n",
       "886    27.0\n",
       "887    19.0\n",
       "888     NaN\n",
       "889    26.0\n",
       "890    32.0\n",
       "Name: Age, Length: 891, dtype: float64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw_data['Age']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      22.000000\n",
       "1      38.000000\n",
       "2      26.000000\n",
       "3      35.000000\n",
       "4      35.000000\n",
       "         ...    \n",
       "886    27.000000\n",
       "887    19.000000\n",
       "888    29.699118\n",
       "889    26.000000\n",
       "890    32.000000\n",
       "Name: Age, Length: 891, dtype: float64"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 填充平均值\n",
    "raw_data['Age'].fillna(value = raw_data['Age'].mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 2. 使用 sklearn的 Imputer函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on class SimpleImputer in module sklearn.impute._base:\n",
      "\n",
      "class SimpleImputer(_BaseImputer)\n",
      " |  SimpleImputer(*, missing_values=nan, strategy='mean', fill_value=None, verbose=0, copy=True, add_indicator=False)\n",
      " |  \n",
      " |  Imputation transformer for completing missing values.\n",
      " |  \n",
      " |  Read more in the :ref:`User Guide <impute>`.\n",
      " |  \n",
      " |  .. versionadded:: 0.20\n",
      " |     `SimpleImputer` replaces the previous `sklearn.preprocessing.Imputer`\n",
      " |     estimator which is now removed.\n",
      " |  \n",
      " |  Parameters\n",
      " |  ----------\n",
      " |  missing_values : number, string, np.nan (default) or None\n",
      " |      The placeholder for the missing values. All occurrences of\n",
      " |      `missing_values` will be imputed. For pandas' dataframes with\n",
      " |      nullable integer dtypes with missing values, `missing_values`\n",
      " |      should be set to `np.nan`, since `pd.NA` will be converted to `np.nan`.\n",
      " |  \n",
      " |  strategy : string, default='mean'\n",
      " |      The imputation strategy.\n",
      " |  \n",
      " |      - If \"mean\", then replace missing values using the mean along\n",
      " |        each column. Can only be used with numeric data.\n",
      " |      - If \"median\", then replace missing values using the median along\n",
      " |        each column. Can only be used with numeric data.\n",
      " |      - If \"most_frequent\", then replace missing using the most frequent\n",
      " |        value along each column. Can be used with strings or numeric data.\n",
      " |      - If \"constant\", then replace missing values with fill_value. Can be\n",
      " |        used with strings or numeric data.\n",
      " |  \n",
      " |      .. versionadded:: 0.20\n",
      " |         strategy=\"constant\" for fixed value imputation.\n",
      " |  \n",
      " |  fill_value : string or numerical value, default=None\n",
      " |      When strategy == \"constant\", fill_value is used to replace all\n",
      " |      occurrences of missing_values.\n",
      " |      If left to the default, fill_value will be 0 when imputing numerical\n",
      " |      data and \"missing_value\" for strings or object data types.\n",
      " |  \n",
      " |  verbose : integer, default=0\n",
      " |      Controls the verbosity of the imputer.\n",
      " |  \n",
      " |  copy : boolean, default=True\n",
      " |      If True, a copy of X will be created. If False, imputation will\n",
      " |      be done in-place whenever possible. Note that, in the following cases,\n",
      " |      a new copy will always be made, even if `copy=False`:\n",
      " |  \n",
      " |      - If X is not an array of floating values;\n",
      " |      - If X is encoded as a CSR matrix;\n",
      " |      - If add_indicator=True.\n",
      " |  \n",
      " |  add_indicator : boolean, default=False\n",
      " |      If True, a :class:`MissingIndicator` transform will stack onto output\n",
      " |      of the imputer's transform. This allows a predictive estimator\n",
      " |      to account for missingness despite imputation. If a feature has no\n",
      " |      missing values at fit/train time, the feature won't appear on\n",
      " |      the missing indicator even if there are missing values at\n",
      " |      transform/test time.\n",
      " |  \n",
      " |  Attributes\n",
      " |  ----------\n",
      " |  statistics_ : array of shape (n_features,)\n",
      " |      The imputation fill value for each feature.\n",
      " |      Computing statistics can result in `np.nan` values.\n",
      " |      During :meth:`transform`, features corresponding to `np.nan`\n",
      " |      statistics will be discarded.\n",
      " |  \n",
      " |  indicator_ : :class:`sklearn.impute.MissingIndicator`\n",
      " |      Indicator used to add binary indicators for missing values.\n",
      " |      ``None`` if add_indicator is False.\n",
      " |  \n",
      " |  See also\n",
      " |  --------\n",
      " |  IterativeImputer : Multivariate imputation of missing values.\n",
      " |  \n",
      " |  Examples\n",
      " |  --------\n",
      " |  >>> import numpy as np\n",
      " |  >>> from sklearn.impute import SimpleImputer\n",
      " |  >>> imp_mean = SimpleImputer(missing_values=np.nan, strategy='mean')\n",
      " |  >>> imp_mean.fit([[7, 2, 3], [4, np.nan, 6], [10, 5, 9]])\n",
      " |  SimpleImputer()\n",
      " |  >>> X = [[np.nan, 2, 3], [4, np.nan, 6], [10, np.nan, 9]]\n",
      " |  >>> print(imp_mean.transform(X))\n",
      " |  [[ 7.   2.   3. ]\n",
      " |   [ 4.   3.5  6. ]\n",
      " |   [10.   3.5  9. ]]\n",
      " |  \n",
      " |  Notes\n",
      " |  -----\n",
      " |  Columns which only contained missing values at :meth:`fit` are discarded\n",
      " |  upon :meth:`transform` if strategy is not \"constant\".\n",
      " |  \n",
      " |  Method resolution order:\n",
      " |      SimpleImputer\n",
      " |      _BaseImputer\n",
      " |      sklearn.base.TransformerMixin\n",
      " |      sklearn.base.BaseEstimator\n",
      " |      builtins.object\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, *, missing_values=nan, strategy='mean', fill_value=None, verbose=0, copy=True, add_indicator=False)\n",
      " |      Initialize self.  See help(type(self)) for accurate signature.\n",
      " |  \n",
      " |  fit(self, X, y=None)\n",
      " |      Fit the imputer on X.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : {array-like, sparse matrix}, shape (n_samples, n_features)\n",
      " |          Input data, where ``n_samples`` is the number of samples and\n",
      " |          ``n_features`` is the number of features.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : SimpleImputer\n",
      " |  \n",
      " |  transform(self, X)\n",
      " |      Impute all missing values in X.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : {array-like, sparse matrix}, shape (n_samples, n_features)\n",
      " |          The input data to complete.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.TransformerMixin:\n",
      " |  \n",
      " |  fit_transform(self, X, y=None, **fit_params)\n",
      " |      Fit to data, then transform it.\n",
      " |      \n",
      " |      Fits transformer to X and y with optional parameters fit_params\n",
      " |      and returns a transformed version of X.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : {array-like, sparse matrix, dataframe} of shape                 (n_samples, n_features)\n",
      " |      \n",
      " |      y : ndarray of shape (n_samples,), default=None\n",
      " |          Target values.\n",
      " |      \n",
      " |      **fit_params : dict\n",
      " |          Additional fit parameters.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      X_new : ndarray array of shape (n_samples, n_features_new)\n",
      " |          Transformed array.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from sklearn.base.TransformerMixin:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.BaseEstimator:\n",
      " |  \n",
      " |  __getstate__(self)\n",
      " |  \n",
      " |  __repr__(self, N_CHAR_MAX=700)\n",
      " |      Return repr(self).\n",
      " |  \n",
      " |  __setstate__(self, state)\n",
      " |  \n",
      " |  get_params(self, deep=True)\n",
      " |      Get parameters for this estimator.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      deep : bool, default=True\n",
      " |          If True, will return the parameters for this estimator and\n",
      " |          contained subobjects that are estimators.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      params : mapping of string to any\n",
      " |          Parameter names mapped to their values.\n",
      " |  \n",
      " |  set_params(self, **params)\n",
      " |      Set the parameters of this estimator.\n",
      " |      \n",
      " |      The method works on simple estimators as well as on nested objects\n",
      " |      (such as pipelines). The latter have parameters of the form\n",
      " |      ``<component>__<parameter>`` so that it's possible to update each\n",
      " |      component of a nested object.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      **params : dict\n",
      " |          Estimator parameters.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : object\n",
      " |          Estimator instance.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.impute import SimpleImputer\n",
    "help(SimpleImputer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[22.        ],\n",
       "       [38.        ],\n",
       "       [26.        ],\n",
       "       [35.        ],\n",
       "       [35.        ],\n",
       "       [29.69911765],\n",
       "       [54.        ],\n",
       "       [ 2.        ],\n",
       "       [27.        ],\n",
       "       [14.        ],\n",
       "       [ 4.        ],\n",
       "       [58.        ],\n",
       "       [20.        ],\n",
       "       [39.        ],\n",
       "       [14.        ],\n",
       "       [55.        ],\n",
       "       [ 2.        ],\n",
       "       [29.69911765],\n",
       "       [31.        ],\n",
       "       [29.69911765],\n",
       "       [35.        ],\n",
       "       [34.        ],\n",
       "       [15.        ],\n",
       "       [28.        ],\n",
       "       [ 8.        ],\n",
       "       [38.        ],\n",
       "       [29.69911765],\n",
       "       [19.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [40.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [66.        ],\n",
       "       [28.        ],\n",
       "       [42.        ],\n",
       "       [29.69911765],\n",
       "       [21.        ],\n",
       "       [18.        ],\n",
       "       [14.        ],\n",
       "       [40.        ],\n",
       "       [27.        ],\n",
       "       [29.69911765],\n",
       "       [ 3.        ],\n",
       "       [19.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [18.        ],\n",
       "       [ 7.        ],\n",
       "       [21.        ],\n",
       "       [49.        ],\n",
       "       [29.        ],\n",
       "       [65.        ],\n",
       "       [29.69911765],\n",
       "       [21.        ],\n",
       "       [28.5       ],\n",
       "       [ 5.        ],\n",
       "       [11.        ],\n",
       "       [22.        ],\n",
       "       [38.        ],\n",
       "       [45.        ],\n",
       "       [ 4.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [29.        ],\n",
       "       [19.        ],\n",
       "       [17.        ],\n",
       "       [26.        ],\n",
       "       [32.        ],\n",
       "       [16.        ],\n",
       "       [21.        ],\n",
       "       [26.        ],\n",
       "       [32.        ],\n",
       "       [25.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [ 0.83      ],\n",
       "       [30.        ],\n",
       "       [22.        ],\n",
       "       [29.        ],\n",
       "       [29.69911765],\n",
       "       [28.        ],\n",
       "       [17.        ],\n",
       "       [33.        ],\n",
       "       [16.        ],\n",
       "       [29.69911765],\n",
       "       [23.        ],\n",
       "       [24.        ],\n",
       "       [29.        ],\n",
       "       [20.        ],\n",
       "       [46.        ],\n",
       "       [26.        ],\n",
       "       [59.        ],\n",
       "       [29.69911765],\n",
       "       [71.        ],\n",
       "       [23.        ],\n",
       "       [34.        ],\n",
       "       [34.        ],\n",
       "       [28.        ],\n",
       "       [29.69911765],\n",
       "       [21.        ],\n",
       "       [33.        ],\n",
       "       [37.        ],\n",
       "       [28.        ],\n",
       "       [21.        ],\n",
       "       [29.69911765],\n",
       "       [38.        ],\n",
       "       [29.69911765],\n",
       "       [47.        ],\n",
       "       [14.5       ],\n",
       "       [22.        ],\n",
       "       [20.        ],\n",
       "       [17.        ],\n",
       "       [21.        ],\n",
       "       [70.5       ],\n",
       "       [29.        ],\n",
       "       [24.        ],\n",
       "       [ 2.        ],\n",
       "       [21.        ],\n",
       "       [29.69911765],\n",
       "       [32.5       ],\n",
       "       [32.5       ],\n",
       "       [54.        ],\n",
       "       [12.        ],\n",
       "       [29.69911765],\n",
       "       [24.        ],\n",
       "       [29.69911765],\n",
       "       [45.        ],\n",
       "       [33.        ],\n",
       "       [20.        ],\n",
       "       [47.        ],\n",
       "       [29.        ],\n",
       "       [25.        ],\n",
       "       [23.        ],\n",
       "       [19.        ],\n",
       "       [37.        ],\n",
       "       [16.        ],\n",
       "       [24.        ],\n",
       "       [29.69911765],\n",
       "       [22.        ],\n",
       "       [24.        ],\n",
       "       [19.        ],\n",
       "       [18.        ],\n",
       "       [19.        ],\n",
       "       [27.        ],\n",
       "       [ 9.        ],\n",
       "       [36.5       ],\n",
       "       [42.        ],\n",
       "       [51.        ],\n",
       "       [22.        ],\n",
       "       [55.5       ],\n",
       "       [40.5       ],\n",
       "       [29.69911765],\n",
       "       [51.        ],\n",
       "       [16.        ],\n",
       "       [30.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [44.        ],\n",
       "       [40.        ],\n",
       "       [26.        ],\n",
       "       [17.        ],\n",
       "       [ 1.        ],\n",
       "       [ 9.        ],\n",
       "       [29.69911765],\n",
       "       [45.        ],\n",
       "       [29.69911765],\n",
       "       [28.        ],\n",
       "       [61.        ],\n",
       "       [ 4.        ],\n",
       "       [ 1.        ],\n",
       "       [21.        ],\n",
       "       [56.        ],\n",
       "       [18.        ],\n",
       "       [29.69911765],\n",
       "       [50.        ],\n",
       "       [30.        ],\n",
       "       [36.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [ 9.        ],\n",
       "       [ 1.        ],\n",
       "       [ 4.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [45.        ],\n",
       "       [40.        ],\n",
       "       [36.        ],\n",
       "       [32.        ],\n",
       "       [19.        ],\n",
       "       [19.        ],\n",
       "       [ 3.        ],\n",
       "       [44.        ],\n",
       "       [58.        ],\n",
       "       [29.69911765],\n",
       "       [42.        ],\n",
       "       [29.69911765],\n",
       "       [24.        ],\n",
       "       [28.        ],\n",
       "       [29.69911765],\n",
       "       [34.        ],\n",
       "       [45.5       ],\n",
       "       [18.        ],\n",
       "       [ 2.        ],\n",
       "       [32.        ],\n",
       "       [26.        ],\n",
       "       [16.        ],\n",
       "       [40.        ],\n",
       "       [24.        ],\n",
       "       [35.        ],\n",
       "       [22.        ],\n",
       "       [30.        ],\n",
       "       [29.69911765],\n",
       "       [31.        ],\n",
       "       [27.        ],\n",
       "       [42.        ],\n",
       "       [32.        ],\n",
       "       [30.        ],\n",
       "       [16.        ],\n",
       "       [27.        ],\n",
       "       [51.        ],\n",
       "       [29.69911765],\n",
       "       [38.        ],\n",
       "       [22.        ],\n",
       "       [19.        ],\n",
       "       [20.5       ],\n",
       "       [18.        ],\n",
       "       [29.69911765],\n",
       "       [35.        ],\n",
       "       [29.        ],\n",
       "       [59.        ],\n",
       "       [ 5.        ],\n",
       "       [24.        ],\n",
       "       [29.69911765],\n",
       "       [44.        ],\n",
       "       [ 8.        ],\n",
       "       [19.        ],\n",
       "       [33.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [29.        ],\n",
       "       [22.        ],\n",
       "       [30.        ],\n",
       "       [44.        ],\n",
       "       [25.        ],\n",
       "       [24.        ],\n",
       "       [37.        ],\n",
       "       [54.        ],\n",
       "       [29.69911765],\n",
       "       [29.        ],\n",
       "       [62.        ],\n",
       "       [30.        ],\n",
       "       [41.        ],\n",
       "       [29.        ],\n",
       "       [29.69911765],\n",
       "       [30.        ],\n",
       "       [35.        ],\n",
       "       [50.        ],\n",
       "       [29.69911765],\n",
       "       [ 3.        ],\n",
       "       [52.        ],\n",
       "       [40.        ],\n",
       "       [29.69911765],\n",
       "       [36.        ],\n",
       "       [16.        ],\n",
       "       [25.        ],\n",
       "       [58.        ],\n",
       "       [35.        ],\n",
       "       [29.69911765],\n",
       "       [25.        ],\n",
       "       [41.        ],\n",
       "       [37.        ],\n",
       "       [29.69911765],\n",
       "       [63.        ],\n",
       "       [45.        ],\n",
       "       [29.69911765],\n",
       "       [ 7.        ],\n",
       "       [35.        ],\n",
       "       [65.        ],\n",
       "       [28.        ],\n",
       "       [16.        ],\n",
       "       [19.        ],\n",
       "       [29.69911765],\n",
       "       [33.        ],\n",
       "       [30.        ],\n",
       "       [22.        ],\n",
       "       [42.        ],\n",
       "       [22.        ],\n",
       "       [26.        ],\n",
       "       [19.        ],\n",
       "       [36.        ],\n",
       "       [24.        ],\n",
       "       [24.        ],\n",
       "       [29.69911765],\n",
       "       [23.5       ],\n",
       "       [ 2.        ],\n",
       "       [29.69911765],\n",
       "       [50.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [19.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [ 0.92      ],\n",
       "       [29.69911765],\n",
       "       [17.        ],\n",
       "       [30.        ],\n",
       "       [30.        ],\n",
       "       [24.        ],\n",
       "       [18.        ],\n",
       "       [26.        ],\n",
       "       [28.        ],\n",
       "       [43.        ],\n",
       "       [26.        ],\n",
       "       [24.        ],\n",
       "       [54.        ],\n",
       "       [31.        ],\n",
       "       [40.        ],\n",
       "       [22.        ],\n",
       "       [27.        ],\n",
       "       [30.        ],\n",
       "       [22.        ],\n",
       "       [29.69911765],\n",
       "       [36.        ],\n",
       "       [61.        ],\n",
       "       [36.        ],\n",
       "       [31.        ],\n",
       "       [16.        ],\n",
       "       [29.69911765],\n",
       "       [45.5       ],\n",
       "       [38.        ],\n",
       "       [16.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [29.        ],\n",
       "       [41.        ],\n",
       "       [45.        ],\n",
       "       [45.        ],\n",
       "       [ 2.        ],\n",
       "       [24.        ],\n",
       "       [28.        ],\n",
       "       [25.        ],\n",
       "       [36.        ],\n",
       "       [24.        ],\n",
       "       [40.        ],\n",
       "       [29.69911765],\n",
       "       [ 3.        ],\n",
       "       [42.        ],\n",
       "       [23.        ],\n",
       "       [29.69911765],\n",
       "       [15.        ],\n",
       "       [25.        ],\n",
       "       [29.69911765],\n",
       "       [28.        ],\n",
       "       [22.        ],\n",
       "       [38.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [40.        ],\n",
       "       [29.        ],\n",
       "       [45.        ],\n",
       "       [35.        ],\n",
       "       [29.69911765],\n",
       "       [30.        ],\n",
       "       [60.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [24.        ],\n",
       "       [25.        ],\n",
       "       [18.        ],\n",
       "       [19.        ],\n",
       "       [22.        ],\n",
       "       [ 3.        ],\n",
       "       [29.69911765],\n",
       "       [22.        ],\n",
       "       [27.        ],\n",
       "       [20.        ],\n",
       "       [19.        ],\n",
       "       [42.        ],\n",
       "       [ 1.        ],\n",
       "       [32.        ],\n",
       "       [35.        ],\n",
       "       [29.69911765],\n",
       "       [18.        ],\n",
       "       [ 1.        ],\n",
       "       [36.        ],\n",
       "       [29.69911765],\n",
       "       [17.        ],\n",
       "       [36.        ],\n",
       "       [21.        ],\n",
       "       [28.        ],\n",
       "       [23.        ],\n",
       "       [24.        ],\n",
       "       [22.        ],\n",
       "       [31.        ],\n",
       "       [46.        ],\n",
       "       [23.        ],\n",
       "       [28.        ],\n",
       "       [39.        ],\n",
       "       [26.        ],\n",
       "       [21.        ],\n",
       "       [28.        ],\n",
       "       [20.        ],\n",
       "       [34.        ],\n",
       "       [51.        ],\n",
       "       [ 3.        ],\n",
       "       [21.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [33.        ],\n",
       "       [29.69911765],\n",
       "       [44.        ],\n",
       "       [29.69911765],\n",
       "       [34.        ],\n",
       "       [18.        ],\n",
       "       [30.        ],\n",
       "       [10.        ],\n",
       "       [29.69911765],\n",
       "       [21.        ],\n",
       "       [29.        ],\n",
       "       [28.        ],\n",
       "       [18.        ],\n",
       "       [29.69911765],\n",
       "       [28.        ],\n",
       "       [19.        ],\n",
       "       [29.69911765],\n",
       "       [32.        ],\n",
       "       [28.        ],\n",
       "       [29.69911765],\n",
       "       [42.        ],\n",
       "       [17.        ],\n",
       "       [50.        ],\n",
       "       [14.        ],\n",
       "       [21.        ],\n",
       "       [24.        ],\n",
       "       [64.        ],\n",
       "       [31.        ],\n",
       "       [45.        ],\n",
       "       [20.        ],\n",
       "       [25.        ],\n",
       "       [28.        ],\n",
       "       [29.69911765],\n",
       "       [ 4.        ],\n",
       "       [13.        ],\n",
       "       [34.        ],\n",
       "       [ 5.        ],\n",
       "       [52.        ],\n",
       "       [36.        ],\n",
       "       [29.69911765],\n",
       "       [30.        ],\n",
       "       [49.        ],\n",
       "       [29.69911765],\n",
       "       [29.        ],\n",
       "       [65.        ],\n",
       "       [29.69911765],\n",
       "       [50.        ],\n",
       "       [29.69911765],\n",
       "       [48.        ],\n",
       "       [34.        ],\n",
       "       [47.        ],\n",
       "       [48.        ],\n",
       "       [29.69911765],\n",
       "       [38.        ],\n",
       "       [29.69911765],\n",
       "       [56.        ],\n",
       "       [29.69911765],\n",
       "       [ 0.75      ],\n",
       "       [29.69911765],\n",
       "       [38.        ],\n",
       "       [33.        ],\n",
       "       [23.        ],\n",
       "       [22.        ],\n",
       "       [29.69911765],\n",
       "       [34.        ],\n",
       "       [29.        ],\n",
       "       [22.        ],\n",
       "       [ 2.        ],\n",
       "       [ 9.        ],\n",
       "       [29.69911765],\n",
       "       [50.        ],\n",
       "       [63.        ],\n",
       "       [25.        ],\n",
       "       [29.69911765],\n",
       "       [35.        ],\n",
       "       [58.        ],\n",
       "       [30.        ],\n",
       "       [ 9.        ],\n",
       "       [29.69911765],\n",
       "       [21.        ],\n",
       "       [55.        ],\n",
       "       [71.        ],\n",
       "       [21.        ],\n",
       "       [29.69911765],\n",
       "       [54.        ],\n",
       "       [29.69911765],\n",
       "       [25.        ],\n",
       "       [24.        ],\n",
       "       [17.        ],\n",
       "       [21.        ],\n",
       "       [29.69911765],\n",
       "       [37.        ],\n",
       "       [16.        ],\n",
       "       [18.        ],\n",
       "       [33.        ],\n",
       "       [29.69911765],\n",
       "       [28.        ],\n",
       "       [26.        ],\n",
       "       [29.        ],\n",
       "       [29.69911765],\n",
       "       [36.        ],\n",
       "       [54.        ],\n",
       "       [24.        ],\n",
       "       [47.        ],\n",
       "       [34.        ],\n",
       "       [29.69911765],\n",
       "       [36.        ],\n",
       "       [32.        ],\n",
       "       [30.        ],\n",
       "       [22.        ],\n",
       "       [29.69911765],\n",
       "       [44.        ],\n",
       "       [29.69911765],\n",
       "       [40.5       ],\n",
       "       [50.        ],\n",
       "       [29.69911765],\n",
       "       [39.        ],\n",
       "       [23.        ],\n",
       "       [ 2.        ],\n",
       "       [29.69911765],\n",
       "       [17.        ],\n",
       "       [29.69911765],\n",
       "       [30.        ],\n",
       "       [ 7.        ],\n",
       "       [45.        ],\n",
       "       [30.        ],\n",
       "       [29.69911765],\n",
       "       [22.        ],\n",
       "       [36.        ],\n",
       "       [ 9.        ],\n",
       "       [11.        ],\n",
       "       [32.        ],\n",
       "       [50.        ],\n",
       "       [64.        ],\n",
       "       [19.        ],\n",
       "       [29.69911765],\n",
       "       [33.        ],\n",
       "       [ 8.        ],\n",
       "       [17.        ],\n",
       "       [27.        ],\n",
       "       [29.69911765],\n",
       "       [22.        ],\n",
       "       [22.        ],\n",
       "       [62.        ],\n",
       "       [48.        ],\n",
       "       [29.69911765],\n",
       "       [39.        ],\n",
       "       [36.        ],\n",
       "       [29.69911765],\n",
       "       [40.        ],\n",
       "       [28.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [24.        ],\n",
       "       [19.        ],\n",
       "       [29.        ],\n",
       "       [29.69911765],\n",
       "       [32.        ],\n",
       "       [62.        ],\n",
       "       [53.        ],\n",
       "       [36.        ],\n",
       "       [29.69911765],\n",
       "       [16.        ],\n",
       "       [19.        ],\n",
       "       [34.        ],\n",
       "       [39.        ],\n",
       "       [29.69911765],\n",
       "       [32.        ],\n",
       "       [25.        ],\n",
       "       [39.        ],\n",
       "       [54.        ],\n",
       "       [36.        ],\n",
       "       [29.69911765],\n",
       "       [18.        ],\n",
       "       [47.        ],\n",
       "       [60.        ],\n",
       "       [22.        ],\n",
       "       [29.69911765],\n",
       "       [35.        ],\n",
       "       [52.        ],\n",
       "       [47.        ],\n",
       "       [29.69911765],\n",
       "       [37.        ],\n",
       "       [36.        ],\n",
       "       [29.69911765],\n",
       "       [49.        ],\n",
       "       [29.69911765],\n",
       "       [49.        ],\n",
       "       [24.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [44.        ],\n",
       "       [35.        ],\n",
       "       [36.        ],\n",
       "       [30.        ],\n",
       "       [27.        ],\n",
       "       [22.        ],\n",
       "       [40.        ],\n",
       "       [39.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [35.        ],\n",
       "       [24.        ],\n",
       "       [34.        ],\n",
       "       [26.        ],\n",
       "       [ 4.        ],\n",
       "       [26.        ],\n",
       "       [27.        ],\n",
       "       [42.        ],\n",
       "       [20.        ],\n",
       "       [21.        ],\n",
       "       [21.        ],\n",
       "       [61.        ],\n",
       "       [57.        ],\n",
       "       [21.        ],\n",
       "       [26.        ],\n",
       "       [29.69911765],\n",
       "       [80.        ],\n",
       "       [51.        ],\n",
       "       [32.        ],\n",
       "       [29.69911765],\n",
       "       [ 9.        ],\n",
       "       [28.        ],\n",
       "       [32.        ],\n",
       "       [31.        ],\n",
       "       [41.        ],\n",
       "       [29.69911765],\n",
       "       [20.        ],\n",
       "       [24.        ],\n",
       "       [ 2.        ],\n",
       "       [29.69911765],\n",
       "       [ 0.75      ],\n",
       "       [48.        ],\n",
       "       [19.        ],\n",
       "       [56.        ],\n",
       "       [29.69911765],\n",
       "       [23.        ],\n",
       "       [29.69911765],\n",
       "       [18.        ],\n",
       "       [21.        ],\n",
       "       [29.69911765],\n",
       "       [18.        ],\n",
       "       [24.        ],\n",
       "       [29.69911765],\n",
       "       [32.        ],\n",
       "       [23.        ],\n",
       "       [58.        ],\n",
       "       [50.        ],\n",
       "       [40.        ],\n",
       "       [47.        ],\n",
       "       [36.        ],\n",
       "       [20.        ],\n",
       "       [32.        ],\n",
       "       [25.        ],\n",
       "       [29.69911765],\n",
       "       [43.        ],\n",
       "       [29.69911765],\n",
       "       [40.        ],\n",
       "       [31.        ],\n",
       "       [70.        ],\n",
       "       [31.        ],\n",
       "       [29.69911765],\n",
       "       [18.        ],\n",
       "       [24.5       ],\n",
       "       [18.        ],\n",
       "       [43.        ],\n",
       "       [36.        ],\n",
       "       [29.69911765],\n",
       "       [27.        ],\n",
       "       [20.        ],\n",
       "       [14.        ],\n",
       "       [60.        ],\n",
       "       [25.        ],\n",
       "       [14.        ],\n",
       "       [19.        ],\n",
       "       [18.        ],\n",
       "       [15.        ],\n",
       "       [31.        ],\n",
       "       [ 4.        ],\n",
       "       [29.69911765],\n",
       "       [25.        ],\n",
       "       [60.        ],\n",
       "       [52.        ],\n",
       "       [44.        ],\n",
       "       [29.69911765],\n",
       "       [49.        ],\n",
       "       [42.        ],\n",
       "       [18.        ],\n",
       "       [35.        ],\n",
       "       [18.        ],\n",
       "       [25.        ],\n",
       "       [26.        ],\n",
       "       [39.        ],\n",
       "       [45.        ],\n",
       "       [42.        ],\n",
       "       [22.        ],\n",
       "       [29.69911765],\n",
       "       [24.        ],\n",
       "       [29.69911765],\n",
       "       [48.        ],\n",
       "       [29.        ],\n",
       "       [52.        ],\n",
       "       [19.        ],\n",
       "       [38.        ],\n",
       "       [27.        ],\n",
       "       [29.69911765],\n",
       "       [33.        ],\n",
       "       [ 6.        ],\n",
       "       [17.        ],\n",
       "       [34.        ],\n",
       "       [50.        ],\n",
       "       [27.        ],\n",
       "       [20.        ],\n",
       "       [30.        ],\n",
       "       [29.69911765],\n",
       "       [25.        ],\n",
       "       [25.        ],\n",
       "       [29.        ],\n",
       "       [11.        ],\n",
       "       [29.69911765],\n",
       "       [23.        ],\n",
       "       [23.        ],\n",
       "       [28.5       ],\n",
       "       [48.        ],\n",
       "       [35.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [36.        ],\n",
       "       [21.        ],\n",
       "       [24.        ],\n",
       "       [31.        ],\n",
       "       [70.        ],\n",
       "       [16.        ],\n",
       "       [30.        ],\n",
       "       [19.        ],\n",
       "       [31.        ],\n",
       "       [ 4.        ],\n",
       "       [ 6.        ],\n",
       "       [33.        ],\n",
       "       [23.        ],\n",
       "       [48.        ],\n",
       "       [ 0.67      ],\n",
       "       [28.        ],\n",
       "       [18.        ],\n",
       "       [34.        ],\n",
       "       [33.        ],\n",
       "       [29.69911765],\n",
       "       [41.        ],\n",
       "       [20.        ],\n",
       "       [36.        ],\n",
       "       [16.        ],\n",
       "       [51.        ],\n",
       "       [29.69911765],\n",
       "       [30.5       ],\n",
       "       [29.69911765],\n",
       "       [32.        ],\n",
       "       [24.        ],\n",
       "       [48.        ],\n",
       "       [57.        ],\n",
       "       [29.69911765],\n",
       "       [54.        ],\n",
       "       [18.        ],\n",
       "       [29.69911765],\n",
       "       [ 5.        ],\n",
       "       [29.69911765],\n",
       "       [43.        ],\n",
       "       [13.        ],\n",
       "       [17.        ],\n",
       "       [29.        ],\n",
       "       [29.69911765],\n",
       "       [25.        ],\n",
       "       [25.        ],\n",
       "       [18.        ],\n",
       "       [ 8.        ],\n",
       "       [ 1.        ],\n",
       "       [46.        ],\n",
       "       [29.69911765],\n",
       "       [16.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [25.        ],\n",
       "       [39.        ],\n",
       "       [49.        ],\n",
       "       [31.        ],\n",
       "       [30.        ],\n",
       "       [30.        ],\n",
       "       [34.        ],\n",
       "       [31.        ],\n",
       "       [11.        ],\n",
       "       [ 0.42      ],\n",
       "       [27.        ],\n",
       "       [31.        ],\n",
       "       [39.        ],\n",
       "       [18.        ],\n",
       "       [39.        ],\n",
       "       [33.        ],\n",
       "       [26.        ],\n",
       "       [39.        ],\n",
       "       [35.        ],\n",
       "       [ 6.        ],\n",
       "       [30.5       ],\n",
       "       [29.69911765],\n",
       "       [23.        ],\n",
       "       [31.        ],\n",
       "       [43.        ],\n",
       "       [10.        ],\n",
       "       [52.        ],\n",
       "       [27.        ],\n",
       "       [38.        ],\n",
       "       [27.        ],\n",
       "       [ 2.        ],\n",
       "       [29.69911765],\n",
       "       [29.69911765],\n",
       "       [ 1.        ],\n",
       "       [29.69911765],\n",
       "       [62.        ],\n",
       "       [15.        ],\n",
       "       [ 0.83      ],\n",
       "       [29.69911765],\n",
       "       [23.        ],\n",
       "       [18.        ],\n",
       "       [39.        ],\n",
       "       [21.        ],\n",
       "       [29.69911765],\n",
       "       [32.        ],\n",
       "       [29.69911765],\n",
       "       [20.        ],\n",
       "       [16.        ],\n",
       "       [30.        ],\n",
       "       [34.5       ],\n",
       "       [17.        ],\n",
       "       [42.        ],\n",
       "       [29.69911765],\n",
       "       [35.        ],\n",
       "       [28.        ],\n",
       "       [29.69911765],\n",
       "       [ 4.        ],\n",
       "       [74.        ],\n",
       "       [ 9.        ],\n",
       "       [16.        ],\n",
       "       [44.        ],\n",
       "       [18.        ],\n",
       "       [45.        ],\n",
       "       [51.        ],\n",
       "       [24.        ],\n",
       "       [29.69911765],\n",
       "       [41.        ],\n",
       "       [21.        ],\n",
       "       [48.        ],\n",
       "       [29.69911765],\n",
       "       [24.        ],\n",
       "       [42.        ],\n",
       "       [27.        ],\n",
       "       [31.        ],\n",
       "       [29.69911765],\n",
       "       [ 4.        ],\n",
       "       [26.        ],\n",
       "       [47.        ],\n",
       "       [33.        ],\n",
       "       [47.        ],\n",
       "       [28.        ],\n",
       "       [15.        ],\n",
       "       [20.        ],\n",
       "       [19.        ],\n",
       "       [29.69911765],\n",
       "       [56.        ],\n",
       "       [25.        ],\n",
       "       [33.        ],\n",
       "       [22.        ],\n",
       "       [28.        ],\n",
       "       [25.        ],\n",
       "       [39.        ],\n",
       "       [27.        ],\n",
       "       [19.        ],\n",
       "       [29.69911765],\n",
       "       [26.        ],\n",
       "       [32.        ]])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置缺失值处理函数，填充年龄缺失值\n",
    "fillNan = SimpleImputer()\n",
    "age = fillNan.fit_transform(raw_data[['Age']].values)\n",
    "age"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      " #   Column       Non-Null Count  Dtype  \n",
      "---  ------       --------------  -----  \n",
      " 0   PassengerId  891 non-null    int64  \n",
      " 1   Survived     891 non-null    int64  \n",
      " 2   Pclass       891 non-null    int64  \n",
      " 3   Name         891 non-null    object \n",
      " 4   Sex          891 non-null    object \n",
      " 5   Age          891 non-null    float64\n",
      " 6   SibSp        891 non-null    int64  \n",
      " 7   Parch        891 non-null    int64  \n",
      " 8   Ticket       891 non-null    object \n",
      " 9   Fare         891 non-null    float64\n",
      " 10  Cabin        204 non-null    object \n",
      " 11  Embarked     889 non-null    object \n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.7+ KB\n"
     ]
    }
   ],
   "source": [
    "# 将处理后的列覆盖到原始数据中\n",
    "raw_data.loc[:, 'Age'] = fillNan.fit_transform(raw_data[['Age']].values)\n",
    "raw_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.特征工程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 数值型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 1.幅度调整\n",
    "- 最大最小值缩放：sklearn的MinMaxScaler函数\n",
    "- 标准化：sklearn的StandardScaler函数\n",
    "- 对数变换：numpy的log函数\n",
    "- 归一化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on class MinMaxScaler in module sklearn.preprocessing._data:\n",
      "\n",
      "class MinMaxScaler(sklearn.base.TransformerMixin, sklearn.base.BaseEstimator)\n",
      " |  MinMaxScaler(feature_range=(0, 1), *, copy=True)\n",
      " |  \n",
      " |  Transform features by scaling each feature to a given range.\n",
      " |  \n",
      " |  This estimator scales and translates each feature individually such\n",
      " |  that it is in the given range on the training set, e.g. between\n",
      " |  zero and one.\n",
      " |  \n",
      " |  The transformation is given by::\n",
      " |  \n",
      " |      X_std = (X - X.min(axis=0)) / (X.max(axis=0) - X.min(axis=0))\n",
      " |      X_scaled = X_std * (max - min) + min\n",
      " |  \n",
      " |  where min, max = feature_range.\n",
      " |  \n",
      " |  This transformation is often used as an alternative to zero mean,\n",
      " |  unit variance scaling.\n",
      " |  \n",
      " |  Read more in the :ref:`User Guide <preprocessing_scaler>`.\n",
      " |  \n",
      " |  Parameters\n",
      " |  ----------\n",
      " |  feature_range : tuple (min, max), default=(0, 1)\n",
      " |      Desired range of transformed data.\n",
      " |  \n",
      " |  copy : bool, default=True\n",
      " |      Set to False to perform inplace row normalization and avoid a\n",
      " |      copy (if the input is already a numpy array).\n",
      " |  \n",
      " |  Attributes\n",
      " |  ----------\n",
      " |  min_ : ndarray of shape (n_features,)\n",
      " |      Per feature adjustment for minimum. Equivalent to\n",
      " |      ``min - X.min(axis=0) * self.scale_``\n",
      " |  \n",
      " |  scale_ : ndarray of shape (n_features,)\n",
      " |      Per feature relative scaling of the data. Equivalent to\n",
      " |      ``(max - min) / (X.max(axis=0) - X.min(axis=0))``\n",
      " |  \n",
      " |      .. versionadded:: 0.17\n",
      " |         *scale_* attribute.\n",
      " |  \n",
      " |  data_min_ : ndarray of shape (n_features,)\n",
      " |      Per feature minimum seen in the data\n",
      " |  \n",
      " |      .. versionadded:: 0.17\n",
      " |         *data_min_*\n",
      " |  \n",
      " |  data_max_ : ndarray of shape (n_features,)\n",
      " |      Per feature maximum seen in the data\n",
      " |  \n",
      " |      .. versionadded:: 0.17\n",
      " |         *data_max_*\n",
      " |  \n",
      " |  data_range_ : ndarray of shape (n_features,)\n",
      " |      Per feature range ``(data_max_ - data_min_)`` seen in the data\n",
      " |  \n",
      " |      .. versionadded:: 0.17\n",
      " |         *data_range_*\n",
      " |  \n",
      " |  n_samples_seen_ : int\n",
      " |      The number of samples processed by the estimator.\n",
      " |      It will be reset on new calls to fit, but increments across\n",
      " |      ``partial_fit`` calls.\n",
      " |  \n",
      " |  Examples\n",
      " |  --------\n",
      " |  >>> from sklearn.preprocessing import MinMaxScaler\n",
      " |  >>> data = [[-1, 2], [-0.5, 6], [0, 10], [1, 18]]\n",
      " |  >>> scaler = MinMaxScaler()\n",
      " |  >>> print(scaler.fit(data))\n",
      " |  MinMaxScaler()\n",
      " |  >>> print(scaler.data_max_)\n",
      " |  [ 1. 18.]\n",
      " |  >>> print(scaler.transform(data))\n",
      " |  [[0.   0.  ]\n",
      " |   [0.25 0.25]\n",
      " |   [0.5  0.5 ]\n",
      " |   [1.   1.  ]]\n",
      " |  >>> print(scaler.transform([[2, 2]]))\n",
      " |  [[1.5 0. ]]\n",
      " |  \n",
      " |  See also\n",
      " |  --------\n",
      " |  minmax_scale: Equivalent function without the estimator API.\n",
      " |  \n",
      " |  Notes\n",
      " |  -----\n",
      " |  NaNs are treated as missing values: disregarded in fit, and maintained in\n",
      " |  transform.\n",
      " |  \n",
      " |  For a comparison of the different scalers, transformers, and normalizers,\n",
      " |  see :ref:`examples/preprocessing/plot_all_scaling.py\n",
      " |  <sphx_glr_auto_examples_preprocessing_plot_all_scaling.py>`.\n",
      " |  \n",
      " |  Method resolution order:\n",
      " |      MinMaxScaler\n",
      " |      sklearn.base.TransformerMixin\n",
      " |      sklearn.base.BaseEstimator\n",
      " |      builtins.object\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, feature_range=(0, 1), *, copy=True)\n",
      " |      Initialize self.  See help(type(self)) for accurate signature.\n",
      " |  \n",
      " |  fit(self, X, y=None)\n",
      " |      Compute the minimum and maximum to be used for later scaling.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like of shape (n_samples, n_features)\n",
      " |          The data used to compute the per-feature minimum and maximum\n",
      " |          used for later scaling along the features axis.\n",
      " |      \n",
      " |      y : None\n",
      " |          Ignored.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : object\n",
      " |          Fitted scaler.\n",
      " |  \n",
      " |  inverse_transform(self, X)\n",
      " |      Undo the scaling of X according to feature_range.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like of shape (n_samples, n_features)\n",
      " |          Input data that will be transformed. It cannot be sparse.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      Xt : array-like of shape (n_samples, n_features)\n",
      " |          Transformed data.\n",
      " |  \n",
      " |  partial_fit(self, X, y=None)\n",
      " |      Online computation of min and max on X for later scaling.\n",
      " |      \n",
      " |      All of X is processed as a single batch. This is intended for cases\n",
      " |      when :meth:`fit` is not feasible due to very large number of\n",
      " |      `n_samples` or because X is read from a continuous stream.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like of shape (n_samples, n_features)\n",
      " |          The data used to compute the mean and standard deviation\n",
      " |          used for later scaling along the features axis.\n",
      " |      \n",
      " |      y : None\n",
      " |          Ignored.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : object\n",
      " |          Transformer instance.\n",
      " |  \n",
      " |  transform(self, X)\n",
      " |      Scale features of X according to feature_range.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like of shape (n_samples, n_features)\n",
      " |          Input data that will be transformed.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      Xt : array-like of shape (n_samples, n_features)\n",
      " |          Transformed data.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.TransformerMixin:\n",
      " |  \n",
      " |  fit_transform(self, X, y=None, **fit_params)\n",
      " |      Fit to data, then transform it.\n",
      " |      \n",
      " |      Fits transformer to X and y with optional parameters fit_params\n",
      " |      and returns a transformed version of X.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : {array-like, sparse matrix, dataframe} of shape                 (n_samples, n_features)\n",
      " |      \n",
      " |      y : ndarray of shape (n_samples,), default=None\n",
      " |          Target values.\n",
      " |      \n",
      " |      **fit_params : dict\n",
      " |          Additional fit parameters.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      X_new : ndarray array of shape (n_samples, n_features_new)\n",
      " |          Transformed array.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from sklearn.base.TransformerMixin:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.BaseEstimator:\n",
      " |  \n",
      " |  __getstate__(self)\n",
      " |  \n",
      " |  __repr__(self, N_CHAR_MAX=700)\n",
      " |      Return repr(self).\n",
      " |  \n",
      " |  __setstate__(self, state)\n",
      " |  \n",
      " |  get_params(self, deep=True)\n",
      " |      Get parameters for this estimator.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      deep : bool, default=True\n",
      " |          If True, will return the parameters for this estimator and\n",
      " |          contained subobjects that are estimators.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      params : mapping of string to any\n",
      " |          Parameter names mapped to their values.\n",
      " |  \n",
      " |  set_params(self, **params)\n",
      " |      Set the parameters of this estimator.\n",
      " |      \n",
      " |      The method works on simple estimators as well as on nested objects\n",
      " |      (such as pipelines). The latter have parameters of the form\n",
      " |      ``<component>__<parameter>`` so that it's possible to update each\n",
      " |      component of a nested object.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      **params : dict\n",
      " |          Estimator parameters.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : object\n",
      " |          Estimator instance.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 最大最小值缩放\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "help(MinMaxScaler)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.01415106],\n",
       "       [0.13913574],\n",
       "       [0.01546857],\n",
       "       [0.1036443 ],\n",
       "       [0.01571255],\n",
       "       [0.0165095 ],\n",
       "       [0.10122886],\n",
       "       [0.04113566],\n",
       "       [0.02173075],\n",
       "       [0.05869429],\n",
       "       [0.03259623],\n",
       "       [0.05182215],\n",
       "       [0.01571255],\n",
       "       [0.06104473],\n",
       "       [0.01533038],\n",
       "       [0.03122992],\n",
       "       [0.05684821],\n",
       "       [0.02537431],\n",
       "       [0.03513366],\n",
       "       [0.01410226],\n",
       "       [0.05074862],\n",
       "       [0.02537431],\n",
       "       [0.01567195],\n",
       "       [0.06929139],\n",
       "       [0.04113566],\n",
       "       [0.06126432],\n",
       "       [0.01410226],\n",
       "       [0.51334181],\n",
       "       [0.01537917],\n",
       "       [0.01541158],\n",
       "       [0.0541074 ],\n",
       "       [0.28598956],\n",
       "       [0.01512699],\n",
       "       [0.02049464],\n",
       "       [0.16038672],\n",
       "       [0.10149724],\n",
       "       [0.01411046],\n",
       "       [0.01571255],\n",
       "       [0.03513366],\n",
       "       [0.02194234],\n",
       "       [0.01849397],\n",
       "       [0.04098927],\n",
       "       [0.01541158],\n",
       "       [0.08115719],\n",
       "       [0.01537917],\n",
       "       [0.01571255],\n",
       "       [0.03025399],\n",
       "       [0.01512699],\n",
       "       [0.04231498],\n",
       "       [0.03474329],\n",
       "       [0.07746484],\n",
       "       [0.01522459],\n",
       "       [0.14976542],\n",
       "       [0.05074862],\n",
       "       [0.12097534],\n",
       "       [0.06929139],\n",
       "       [0.02049464],\n",
       "       [0.01411046],\n",
       "       [0.05416439],\n",
       "       [0.0915427 ],\n",
       "       [0.01411046],\n",
       "       [0.1561496 ],\n",
       "       [0.16293235],\n",
       "       [0.05445717],\n",
       "       [0.0541074 ],\n",
       "       [0.02975782],\n",
       "       [0.02049464],\n",
       "       [0.01592394],\n",
       "       [0.01546857],\n",
       "       [0.01690807],\n",
       "       [0.02049464],\n",
       "       [0.0915427 ],\n",
       "       [0.14346245],\n",
       "       [0.02821272],\n",
       "       [0.11027246],\n",
       "       [0.01493181],\n",
       "       [0.01541158],\n",
       "       [0.01571255],\n",
       "       [0.05660423],\n",
       "       [0.02434958],\n",
       "       [0.01756683],\n",
       "       [0.01854277],\n",
       "       [0.01520019],\n",
       "       [0.09193308],\n",
       "       [0.02049464],\n",
       "       [0.03093714],\n",
       "       [0.06709553],\n",
       "       [0.01571255],\n",
       "       [0.51334181],\n",
       "       [0.01571255],\n",
       "       [0.01571255],\n",
       "       [0.01533038],\n",
       "       [0.11940565],\n",
       "       [0.04015973],\n",
       "       [0.01415106],\n",
       "       [0.01571255],\n",
       "       [0.06764049],\n",
       "       [0.12366717],\n",
       "       [0.04489301],\n",
       "       [0.05074862],\n",
       "       [0.01541158],\n",
       "       [0.01541158],\n",
       "       [0.15085515],\n",
       "       [0.01689187],\n",
       "       [0.01546857],\n",
       "       [0.01541158],\n",
       "       [0.01493181],\n",
       "       [0.01517579],\n",
       "       [0.01541158],\n",
       "       [0.04713766],\n",
       "       [0.10149724],\n",
       "       [0.02821272],\n",
       "       [0.01571255],\n",
       "       [0.01917712],\n",
       "       [0.02822072],\n",
       "       [0.01546857],\n",
       "       [0.01512699],\n",
       "       [0.04098927],\n",
       "       [0.48312843],\n",
       "       [0.06104473],\n",
       "       [0.14346245],\n",
       "       [0.01571255],\n",
       "       [0.05869429],\n",
       "       [0.02537431],\n",
       "       [0.15085515],\n",
       "       [0.02194234],\n",
       "       [0.01512699],\n",
       "       [0.01393967],\n",
       "       [0.0436405 ],\n",
       "       [0.01361429],\n",
       "       [0.01541158],\n",
       "       [0.01376068],\n",
       "       [0.02830212],\n",
       "       [0.05074862],\n",
       "       [0.02537431],\n",
       "       [0.02936745],\n",
       "       [0.05130158],\n",
       "       [0.1036443 ],\n",
       "       [0.0179898 ],\n",
       "       [0.1545881 ],\n",
       "       [0.02975782],\n",
       "       [0.01512699],\n",
       "       [0.03093714],\n",
       "       [0.01317512],\n",
       "       [0.02244651],\n",
       "       [0.07173122],\n",
       "       [0.01521639],\n",
       "       [0.06709553],\n",
       "       [0.05074862],\n",
       "       [0.02537431],\n",
       "       [0.02444717],\n",
       "       [0.12999454],\n",
       "       [0.01571255],\n",
       "       [0.02830212],\n",
       "       [0.01427305],\n",
       "       [0.11980422],\n",
       "       [0.0150944 ],\n",
       "       [0.01571255],\n",
       "       [0.01690807],\n",
       "       [0.13575256],\n",
       "       [0.03142511],\n",
       "       [0.03074195],\n",
       "       [0.01517579],\n",
       "       [0.01690807],\n",
       "       [0.07746484],\n",
       "       [0.04006213],\n",
       "       [0.10735285],\n",
       "       [0.05445717],\n",
       "       [0.05060223],\n",
       "       [0.11027246],\n",
       "       [0.06538765],\n",
       "       [0.05684821],\n",
       "       [0.02173075],\n",
       "       [0.01546857],\n",
       "       [0.05991421],\n",
       "       [0.01533038],\n",
       "       [0.04970769],\n",
       "       [0.05604307],\n",
       "       [0.02537431],\n",
       "       [0.        ],\n",
       "       [0.13575256],\n",
       "       [0.02937564],\n",
       "       [0.06126432],\n",
       "       [0.07612293],\n",
       "       [0.04298994],\n",
       "       [0.0975935 ],\n",
       "       [0.03025399],\n",
       "       [0.05182215],\n",
       "       [0.03025399],\n",
       "       [0.01541158],\n",
       "       [0.02537431],\n",
       "       [0.02537431],\n",
       "       [0.01533038],\n",
       "       [0.05074862],\n",
       "       [0.0541074 ],\n",
       "       [0.28598956],\n",
       "       [0.01512699],\n",
       "       [0.01640391],\n",
       "       [0.01512699],\n",
       "       [0.02537431],\n",
       "       [0.01854277],\n",
       "       [0.13575256],\n",
       "       [0.01267896],\n",
       "       [0.01410226],\n",
       "       [0.01571255],\n",
       "       [0.02042144],\n",
       "       [0.03093714],\n",
       "       [0.03667076],\n",
       "       [0.01512699],\n",
       "       [0.06050797],\n",
       "       [0.01376068],\n",
       "       [0.04098927],\n",
       "       [0.01415106],\n",
       "       [0.02537431],\n",
       "       [0.01512699],\n",
       "       [0.22109808],\n",
       "       [0.01546857],\n",
       "       [0.05270049],\n",
       "       [0.14891148],\n",
       "       [0.02049464],\n",
       "       [0.01571255],\n",
       "       [0.02537431],\n",
       "       [0.01571255],\n",
       "       [0.01541158],\n",
       "       [0.1756683 ],\n",
       "       [0.01824998],\n",
       "       [0.02049464],\n",
       "       [0.01415106],\n",
       "       [0.02537431],\n",
       "       [0.04970769],\n",
       "       [0.16293235],\n",
       "       [0.01517579],\n",
       "       [0.02635025],\n",
       "       [0.06126432],\n",
       "       [0.02049464],\n",
       "       [0.01473662],\n",
       "       [0.05074862],\n",
       "       [0.05123659],\n",
       "       [0.02049464],\n",
       "       [0.0239592 ],\n",
       "       [0.02821272],\n",
       "       [0.03025399],\n",
       "       [0.02049464],\n",
       "       [0.01390707],\n",
       "       [0.01410226],\n",
       "       [0.1756683 ],\n",
       "       [0.01517579],\n",
       "       [0.02830212],\n",
       "       [0.10257897],\n",
       "       [0.05074862],\n",
       "       [0.01415106],\n",
       "       [0.02042144],\n",
       "       [0.05182215],\n",
       "       [0.03142511],\n",
       "       [0.03945217],\n",
       "       [0.02975782],\n",
       "       [0.1545881 ],\n",
       "       [0.16883676],\n",
       "       [1.        ],\n",
       "       [0.05074862],\n",
       "       [0.01512699],\n",
       "       [0.06126432],\n",
       "       [0.15546645],\n",
       "       [0.        ],\n",
       "       [0.01512699],\n",
       "       [0.02049464],\n",
       "       [0.07746484],\n",
       "       [0.01517579],\n",
       "       [0.29953885],\n",
       "       [0.26473857],\n",
       "       [0.06050797],\n",
       "       [0.        ],\n",
       "       [0.03806147],\n",
       "       [0.05797054],\n",
       "       [0.01512699],\n",
       "       [0.15216447],\n",
       "       [0.01512699],\n",
       "       [0.        ],\n",
       "       [0.05684821],\n",
       "       [0.03952537],\n",
       "       [0.01512699],\n",
       "       [0.01533038],\n",
       "       [0.01854277],\n",
       "       [0.01571255],\n",
       "       [0.05074862],\n",
       "       [0.01690807],\n",
       "       [0.01854277],\n",
       "       [0.01541158],\n",
       "       [0.02537431],\n",
       "       [0.01512699],\n",
       "       [0.15390495],\n",
       "       [0.17777476],\n",
       "       [0.02513033],\n",
       "       [0.01727405],\n",
       "       [0.01541158],\n",
       "       [0.0541074 ],\n",
       "       [0.01411046],\n",
       "       [0.2958059 ],\n",
       "       [0.05953204],\n",
       "       [0.48312843],\n",
       "       [0.01512699],\n",
       "       [0.04538098],\n",
       "       [0.        ],\n",
       "       [0.02410559],\n",
       "       [0.01571255],\n",
       "       [0.2958059 ],\n",
       "       [0.21642979],\n",
       "       [0.21255864],\n",
       "       [0.04684488],\n",
       "       [0.1111184 ],\n",
       "       [0.16231419],\n",
       "       [0.51212189],\n",
       "       [0.05074862],\n",
       "       [0.01541158],\n",
       "       [0.05123659],\n",
       "       [0.01533038],\n",
       "       [0.05074862],\n",
       "       [0.02732618],\n",
       "       [0.32179837],\n",
       "       [0.26252652],\n",
       "       [0.01415106],\n",
       "       [0.01541158],\n",
       "       [0.02410559],\n",
       "       [0.05660423],\n",
       "       [0.13575256],\n",
       "       [0.26473857],\n",
       "       [0.01217479],\n",
       "       [0.02537431],\n",
       "       [0.04006213],\n",
       "       [0.11316786],\n",
       "       [0.04538098],\n",
       "       [0.0556283 ],\n",
       "       [0.29953885],\n",
       "       [0.03513366],\n",
       "       [0.26086743],\n",
       "       [0.01541158],\n",
       "       [0.12999454],\n",
       "       [0.26252652],\n",
       "       [0.01571255],\n",
       "       [0.06929139],\n",
       "       [0.05074862],\n",
       "       [0.51334181],\n",
       "       [0.02537431],\n",
       "       [0.02537431],\n",
       "       [0.02537431],\n",
       "       [0.02537431],\n",
       "       [0.02537431],\n",
       "       [0.03142511],\n",
       "       [0.03103473],\n",
       "       [0.01690807],\n",
       "       [0.018006  ],\n",
       "       [0.06831545],\n",
       "       [0.01411046],\n",
       "       [0.03474329],\n",
       "       [0.01410226],\n",
       "       [0.01854277],\n",
       "       [0.10735285],\n",
       "       [0.02537431],\n",
       "       [0.01537917],\n",
       "       [0.01537917],\n",
       "       [0.05445717],\n",
       "       [0.0541074 ],\n",
       "       [0.02821272],\n",
       "       [0.01376068],\n",
       "       [0.03025399],\n",
       "       [0.01415106],\n",
       "       [0.14687822],\n",
       "       [0.01411046],\n",
       "       [0.01512699],\n",
       "       [0.13526459],\n",
       "       [0.10821499],\n",
       "       [0.01267896],\n",
       "       [0.01571255],\n",
       "       [0.26473857],\n",
       "       [0.04113566],\n",
       "       [0.16038672],\n",
       "       [0.01415106],\n",
       "       [0.41282051],\n",
       "       [0.00783188],\n",
       "       [0.01517579],\n",
       "       [0.44409922],\n",
       "       [0.03072575],\n",
       "       [0.01546857],\n",
       "       [0.10149724],\n",
       "       [0.01541158],\n",
       "       [0.14346245],\n",
       "       [0.0915427 ],\n",
       "       [0.02537431],\n",
       "       [0.01508639],\n",
       "       [0.02342244],\n",
       "       [0.2342244 ],\n",
       "       [0.01521639],\n",
       "       [0.01546857],\n",
       "       [0.22109808],\n",
       "       [0.03259623],\n",
       "       [0.01521639],\n",
       "       [0.01533038],\n",
       "       [0.05074862],\n",
       "       [0.02049464],\n",
       "       [0.02469116],\n",
       "       [0.01546857],\n",
       "       [0.01571255],\n",
       "       [0.01917712],\n",
       "       [0.03093714],\n",
       "       [0.01690807],\n",
       "       [0.04098927],\n",
       "       [0.01512699],\n",
       "       [0.03659756],\n",
       "       [0.01517579],\n",
       "       [0.04970769],\n",
       "       [0.01541158],\n",
       "       [0.01338651],\n",
       "       [0.1756683 ],\n",
       "       [0.        ],\n",
       "       [0.01546857],\n",
       "       [0.01571255],\n",
       "       [0.06343578],\n",
       "       [0.02537431],\n",
       "       [0.02537431],\n",
       "       [0.04713766],\n",
       "       [0.01541158],\n",
       "       [0.0150944 ],\n",
       "       [0.01537098],\n",
       "       [0.02810693],\n",
       "       [0.03945217],\n",
       "       [0.01415106],\n",
       "       [0.05074862],\n",
       "       [0.05074862],\n",
       "       [0.01512699],\n",
       "       [0.01571255],\n",
       "       [0.05182215],\n",
       "       [0.03142511],\n",
       "       [0.05074862],\n",
       "       [0.01390707],\n",
       "       [0.10910953],\n",
       "       [0.2342244 ],\n",
       "       [0.06709553],\n",
       "       [0.03659756],\n",
       "       [0.51334181],\n",
       "       [0.02049464],\n",
       "       [0.05123659],\n",
       "       [0.01854277],\n",
       "       [0.01517579],\n",
       "       [0.02537431],\n",
       "       [0.01583455],\n",
       "       [0.15977676],\n",
       "       [0.03806147],\n",
       "       [0.05182215],\n",
       "       [0.0375897 ],\n",
       "       [0.05953204],\n",
       "       [0.05416439],\n",
       "       [0.0389724 ],\n",
       "       [0.05416439],\n",
       "       [0.17391982],\n",
       "       [0.01571255],\n",
       "       [0.01541158],\n",
       "       [0.05182215],\n",
       "       [0.10122886],\n",
       "       [0.02049464],\n",
       "       [0.01512699],\n",
       "       [0.05182215],\n",
       "       [0.01571255],\n",
       "       [0.075147  ],\n",
       "       [0.02537431],\n",
       "       [0.01571255],\n",
       "       [0.01376068],\n",
       "       [0.        ],\n",
       "       [0.05182215],\n",
       "       [0.0150782 ],\n",
       "       [0.0375897 ],\n",
       "       [0.01415106],\n",
       "       [0.01690807],\n",
       "       [0.05416439],\n",
       "       [0.02691961],\n",
       "       [0.01920152],\n",
       "       [0.10149724],\n",
       "       [0.04098927],\n",
       "       [0.01375249],\n",
       "       [0.01467962],\n",
       "       [0.0239836 ],\n",
       "       [0.0915427 ],\n",
       "       [0.        ],\n",
       "       [0.01571255],\n",
       "       [0.01871355],\n",
       "       [0.17777476],\n",
       "       [0.04970769],\n",
       "       [0.1756683 ],\n",
       "       [0.05797054],\n",
       "       [0.01571255],\n",
       "       [0.03103473],\n",
       "       [0.0389724 ],\n",
       "       [0.01415106],\n",
       "       [0.05953204],\n",
       "       [0.09662576],\n",
       "       [0.01571255],\n",
       "       [0.02822072],\n",
       "       [0.15276642],\n",
       "       [0.02947324],\n",
       "       [0.2958059 ],\n",
       "       [0.01521639],\n",
       "       [0.01690807],\n",
       "       [0.01512699],\n",
       "       [0.01489121],\n",
       "       [0.01871355],\n",
       "       [0.16883676],\n",
       "       [0.21255864],\n",
       "       [0.05074862],\n",
       "       [0.05182215],\n",
       "       [0.04396587],\n",
       "       [0.11027246],\n",
       "       [0.01512699],\n",
       "       [0.01571255],\n",
       "       [0.05130978],\n",
       "       [0.11594108],\n",
       "       [0.01463083],\n",
       "       [0.06640418],\n",
       "       [0.02049464],\n",
       "       [0.04713766],\n",
       "       [0.05074862],\n",
       "       [0.01541158],\n",
       "       [0.18249985],\n",
       "       [0.01541158],\n",
       "       [0.01410226],\n",
       "       [0.11316786],\n",
       "       [0.01411046],\n",
       "       [0.01512699],\n",
       "       [0.02049464],\n",
       "       [0.43288417],\n",
       "       [0.01546857],\n",
       "       [0.02244651],\n",
       "       [0.05074862],\n",
       "       [0.01411046],\n",
       "       [0.01411046],\n",
       "       [0.0436405 ],\n",
       "       [0.01690807],\n",
       "       [0.05123659],\n",
       "       [0.05182215],\n",
       "       [0.20772777],\n",
       "       [0.02830212],\n",
       "       [0.09661757],\n",
       "       [0.13858277],\n",
       "       [0.06104473],\n",
       "       [0.06104473],\n",
       "       [0.05074862],\n",
       "       [0.20772777],\n",
       "       [0.05074862],\n",
       "       [0.05074862],\n",
       "       [0.0270578 ],\n",
       "       [0.04006213],\n",
       "       [0.07173122],\n",
       "       [0.21642979],\n",
       "       [0.05074862],\n",
       "       [0.01528158],\n",
       "       [0.01410226],\n",
       "       [0.01517579],\n",
       "       [0.05182215],\n",
       "       [0.07729405],\n",
       "       [0.44409922],\n",
       "       [0.15546645],\n",
       "       [0.03396254],\n",
       "       [0.01512699],\n",
       "       [0.01541158],\n",
       "       [0.02635025],\n",
       "       [0.01571255],\n",
       "       [0.01571255],\n",
       "       [0.04713766],\n",
       "       [0.01541158],\n",
       "       [0.04113566],\n",
       "       [0.01411046],\n",
       "       [0.01533038],\n",
       "       [0.02049464],\n",
       "       [0.10048071],\n",
       "       [0.05150497],\n",
       "       [0.01512699],\n",
       "       [0.01571255],\n",
       "       [0.02830212],\n",
       "       [0.02537431],\n",
       "       [0.10910953],\n",
       "       [0.02822072],\n",
       "       [0.01546857],\n",
       "       [0.0585561 ],\n",
       "       [0.21642979],\n",
       "       [0.05074862],\n",
       "       [0.07831878],\n",
       "       [0.01700567],\n",
       "       [0.15546645],\n",
       "       [0.02927805],\n",
       "       [0.1545881 ],\n",
       "       [0.01571255],\n",
       "       [0.01571255],\n",
       "       [0.01390707],\n",
       "       [0.15276642],\n",
       "       [0.01415106],\n",
       "       [0.01512699],\n",
       "       [0.05074862],\n",
       "       [0.04713766],\n",
       "       [0.06441171],\n",
       "       [0.        ],\n",
       "       [0.01410226],\n",
       "       [0.1111184 ],\n",
       "       [0.05270049],\n",
       "       [0.01541158],\n",
       "       [0.08275929],\n",
       "       [0.01571255],\n",
       "       [0.05182215],\n",
       "       [0.03035158],\n",
       "       [0.01541158],\n",
       "       [0.05953204],\n",
       "       [0.08115719],\n",
       "       [0.29953885],\n",
       "       [0.06104473],\n",
       "       [0.01376068],\n",
       "       [0.03025399],\n",
       "       [0.01512699],\n",
       "       [0.01571255],\n",
       "       [0.12687155],\n",
       "       [0.02810693],\n",
       "       [0.03142511],\n",
       "       [0.07612293],\n",
       "       [0.02049464],\n",
       "       [0.02821272],\n",
       "       [0.10257897],\n",
       "       [0.03072575],\n",
       "       [0.01533038],\n",
       "       [0.03142511],\n",
       "       [0.063086  ],\n",
       "       [0.02410559],\n",
       "       [0.15216447],\n",
       "       [0.01541158],\n",
       "       [0.0150944 ],\n",
       "       [0.0585561 ],\n",
       "       [0.01376888],\n",
       "       [0.05953204],\n",
       "       [0.        ],\n",
       "       [0.05445717],\n",
       "       [0.02537431],\n",
       "       [0.01546857],\n",
       "       [0.05123659],\n",
       "       [0.07746484],\n",
       "       [0.03142511],\n",
       "       [0.01533038],\n",
       "       [0.13526459],\n",
       "       [0.05445717],\n",
       "       [0.11027246],\n",
       "       [0.0375897 ],\n",
       "       [0.14976542],\n",
       "       [0.01541158],\n",
       "       [0.06929139],\n",
       "       [0.01473662],\n",
       "       [0.01473662],\n",
       "       [0.01541158],\n",
       "       [0.04489301],\n",
       "       [0.01646071],\n",
       "       [0.01528158],\n",
       "       [0.01317512],\n",
       "       [0.14346245],\n",
       "       [0.01541158],\n",
       "       [0.03025399],\n",
       "       [0.02537431],\n",
       "       [0.22109808],\n",
       "       [0.26086743],\n",
       "       [0.01410226],\n",
       "       [0.04994347],\n",
       "       [0.01463083],\n",
       "       [0.01546857],\n",
       "       [0.14346245],\n",
       "       [0.02537431],\n",
       "       [0.01517579],\n",
       "       [0.01571255],\n",
       "       [0.10149724],\n",
       "       [0.07612293],\n",
       "       [0.10149724],\n",
       "       [0.02049464],\n",
       "       [0.02537431],\n",
       "       [0.        ],\n",
       "       [0.01517579],\n",
       "       [0.01571255],\n",
       "       [0.01920972],\n",
       "       [0.0915427 ],\n",
       "       [1.        ],\n",
       "       [0.01588334],\n",
       "       [0.14976542],\n",
       "       [0.018006  ],\n",
       "       [0.0915427 ],\n",
       "       [0.07612293],\n",
       "       [0.08115719],\n",
       "       [0.07746484],\n",
       "       [0.01985208],\n",
       "       [0.01521639],\n",
       "       [0.41250333],\n",
       "       [0.11125659],\n",
       "       [0.02618765],\n",
       "       [0.11027246],\n",
       "       [0.01410226],\n",
       "       [0.05182215],\n",
       "       [0.02635025],\n",
       "       [0.01571255],\n",
       "       [0.0150944 ],\n",
       "       [0.21642979],\n",
       "       [0.01493181],\n",
       "       [0.44409922],\n",
       "       [0.05130978],\n",
       "       [0.02821272],\n",
       "       [0.01511079],\n",
       "       [0.01533038],\n",
       "       [0.05074862],\n",
       "       [0.02635025],\n",
       "       [0.05130978],\n",
       "       [0.2958059 ],\n",
       "       [0.02975782],\n",
       "       [0.09662576],\n",
       "       [0.05182215],\n",
       "       [0.10149724],\n",
       "       [0.01851017],\n",
       "       [0.02537431],\n",
       "       [0.01493181],\n",
       "       [0.44409922],\n",
       "       [0.02049464],\n",
       "       [0.03025399],\n",
       "       [0.01517579],\n",
       "       [0.06441171],\n",
       "       [0.01376888],\n",
       "       [0.02537431],\n",
       "       [0.02537431],\n",
       "       [0.1036443 ],\n",
       "       [0.01690807],\n",
       "       [0.04098927],\n",
       "       [0.01510259],\n",
       "       [0.05074862],\n",
       "       [0.01546857],\n",
       "       [0.41250333],\n",
       "       [0.03667076],\n",
       "       [0.        ],\n",
       "       [0.02537431],\n",
       "       [0.02537431],\n",
       "       [0.03142511],\n",
       "       [0.06709553],\n",
       "       [1.        ],\n",
       "       [0.01541158],\n",
       "       [0.01541158],\n",
       "       [0.0585561 ],\n",
       "       [0.15390495],\n",
       "       [0.51212189],\n",
       "       [0.03142511],\n",
       "       [0.01546857],\n",
       "       [0.13858277],\n",
       "       [0.03952537],\n",
       "       [0.02537431],\n",
       "       [0.1036443 ],\n",
       "       [0.01512699],\n",
       "       [0.04489301],\n",
       "       [0.02434958],\n",
       "       [0.01854277],\n",
       "       [0.01541158],\n",
       "       [0.12687155],\n",
       "       [0.02830212],\n",
       "       [0.01521639],\n",
       "       [0.02244651],\n",
       "       [0.01571255],\n",
       "       [0.16883676],\n",
       "       [0.02830212],\n",
       "       [0.01390707],\n",
       "       [0.01411046],\n",
       "       [0.2342244 ],\n",
       "       [0.01517579],\n",
       "       [0.15216447],\n",
       "       [0.07729405],\n",
       "       [0.01512699],\n",
       "       [0.04713766],\n",
       "       [0.01632251],\n",
       "       [0.01854277],\n",
       "       [0.01533038],\n",
       "       [0.02049464],\n",
       "       [0.01410226],\n",
       "       [0.04489301],\n",
       "       [0.01512699],\n",
       "       [0.01512699],\n",
       "       [0.02434958],\n",
       "       [0.01510259],\n",
       "       [0.41250333],\n",
       "       [0.01411046],\n",
       "       [0.11125659],\n",
       "       [0.0585561 ],\n",
       "       [0.04577135],\n",
       "       [0.01376068],\n",
       "       [0.01415106],\n",
       "       [0.01463083],\n",
       "       [0.05684821],\n",
       "       [0.04015973],\n",
       "       [0.1545881 ],\n",
       "       [0.01512699],\n",
       "       [0.05074862],\n",
       "       [0.13575256],\n",
       "       [0.05991421],\n",
       "       [0.01541158],\n",
       "       [0.02537431],\n",
       "       [0.05061043],\n",
       "       [0.01694867],\n",
       "       [0.01411046],\n",
       "       [0.04713766],\n",
       "       [0.02537431],\n",
       "       [0.05123659],\n",
       "       [0.2342244 ],\n",
       "       [0.01662349],\n",
       "       [0.01361429],\n",
       "       [0.01517579],\n",
       "       [0.        ],\n",
       "       [0.01517579],\n",
       "       [0.02537431],\n",
       "       [0.1036443 ],\n",
       "       [0.01539537],\n",
       "       [0.04713766],\n",
       "       [0.02049464],\n",
       "       [0.06104473],\n",
       "       [0.01571255],\n",
       "       [0.        ],\n",
       "       [0.01546857],\n",
       "       [0.07222739],\n",
       "       [0.01258956],\n",
       "       [0.05445717],\n",
       "       [0.18249985],\n",
       "       [0.01690807],\n",
       "       [0.        ],\n",
       "       [0.02434958],\n",
       "       [0.07746484],\n",
       "       [0.0135655 ],\n",
       "       [0.11027246],\n",
       "       [0.07222739],\n",
       "       [0.01512699],\n",
       "       [0.1561496 ],\n",
       "       [0.02821272],\n",
       "       [0.03659756],\n",
       "       [0.01411046],\n",
       "       [0.01533038],\n",
       "       [0.01620052],\n",
       "       [0.16231419],\n",
       "       [0.01690807],\n",
       "       [0.01571255],\n",
       "       [0.11027246],\n",
       "       [0.05797054],\n",
       "       [0.01546857],\n",
       "       [0.02049464],\n",
       "       [0.06050797],\n",
       "       [0.01256516],\n",
       "       [0.01690807],\n",
       "       [0.01473662],\n",
       "       [0.13575256],\n",
       "       [0.01541158],\n",
       "       [0.06441171],\n",
       "       [0.17391982],\n",
       "       [0.06104473],\n",
       "       [0.01517579],\n",
       "       [0.02975782],\n",
       "       [0.07690368],\n",
       "       [0.05074862],\n",
       "       [0.01824998],\n",
       "       [0.32179837],\n",
       "       [0.05182215],\n",
       "       [0.0375897 ],\n",
       "       [0.01411046],\n",
       "       [0.02753757],\n",
       "       [0.02244651],\n",
       "       [0.05061043],\n",
       "       [0.13575256],\n",
       "       [0.02537431],\n",
       "       [0.02537431],\n",
       "       [0.0270496 ],\n",
       "       [0.09856124],\n",
       "       [0.01854277],\n",
       "       [0.02173075],\n",
       "       [0.01541158],\n",
       "       [0.10257897],\n",
       "       [0.00975935],\n",
       "       [0.01756683],\n",
       "       [0.04684488],\n",
       "       [0.01410226],\n",
       "       [0.01921772],\n",
       "       [0.01541158],\n",
       "       [0.01541158],\n",
       "       [0.16231419],\n",
       "       [0.05074862],\n",
       "       [0.01541158],\n",
       "       [0.02052723],\n",
       "       [0.02049464],\n",
       "       [0.01376068],\n",
       "       [0.05684821],\n",
       "       [0.02537431],\n",
       "       [0.0585561 ],\n",
       "       [0.04577135],\n",
       "       [0.0585561 ],\n",
       "       [0.01512699]])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "minmaxScaler = MinMaxScaler()\n",
    "minmaxScaler.fit_transform(raw_data[['Fare']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>-0.502445</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass                     Name   Sex   Age  SibSp  \\\n",
       "0            1         0       3  Braund, Mr. Owen Harris  male  22.0      1   \n",
       "\n",
       "   Parch     Ticket      Fare Cabin Embarked  \n",
       "0      0  A/5 21171 -0.502445   NaN        S  "
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 标准化\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "standardScaler = StandardScaler()\n",
    "fare = standardScaler.fit_transform(raw_data[['Fare']])\n",
    "# 将标准化数据覆盖到原始数据集\n",
    "raw_data.loc[:, 'Fare'] = fare\n",
    "raw_data.head(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>3.091042</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>-0.502445</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass                     Name   Sex       Age  \\\n",
       "0            1         0       3  Braund, Mr. Owen Harris  male  3.091042   \n",
       "\n",
       "   SibSp  Parch     Ticket      Fare Cabin Embarked  \n",
       "0      1      0  A/5 21171 -0.502445   NaN        S  "
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 对数变换\n",
    "import numpy as np\n",
    "age_log = raw_data['Age'].apply(lambda x:np.log(x))\n",
    "# 将年龄字段做对数变换后，覆盖到原始数据集\n",
    "raw_data.loc[:, 'Age'] = age_log\n",
    "raw_data.head(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 2.统计值\n",
    "- 最大值\n",
    "- 最小值\n",
    "- 分位数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.382026634673881"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "age_log_max = raw_data['Age'].max()\n",
    "age_log_max"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.8675005677047231"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "age_log_min = raw_data['Age'].min()\n",
    "age_log_min"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.091042453358316"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 1/4 位置的数值\n",
    "raw_data['Age'].quantile(0.25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.分箱/分桶/离散化\n",
    "- 等距划分：pandas cut\n",
    "- 等频划分：pandas qcut"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>fare_cut</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>3.091042</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>-0.502445</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>(-0.659, 1.415]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>3.637586</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>0.786845</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>(-0.659, 1.415]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "\n",
       "                                                Name     Sex       Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  3.091042      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  3.637586      1   \n",
       "\n",
       "   Parch     Ticket      Fare Cabin Embarked         fare_cut  \n",
       "0      0  A/5 21171 -0.502445   NaN        S  (-0.659, 1.415]  \n",
       "1      0   PC 17599  0.786845   C85        C  (-0.659, 1.415]  "
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 等距切为 5 份\n",
    "raw_data.loc[:, 'fare_cut'] = pd.cut(raw_data['Fare'], 5)\n",
    "raw_data.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>fare_cut</th>\n",
       "      <th>fare_qcut</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>3.091042</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>-0.502445</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>(-0.659, 1.415]</td>\n",
       "      <td>(-0.649, -0.49]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>3.637586</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>0.786845</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>(-0.659, 1.415]</td>\n",
       "      <td>(0.151, 9.667]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "\n",
       "                                                Name     Sex       Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  3.091042      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  3.637586      1   \n",
       "\n",
       "   Parch     Ticket      Fare Cabin Embarked         fare_cut        fare_qcut  \n",
       "0      0  A/5 21171 -0.502445   NaN        S  (-0.659, 1.415]  (-0.649, -0.49]  \n",
       "1      0   PC 17599  0.786845   C85        C  (-0.659, 1.415]   (0.151, 9.667]  "
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 等频划分\n",
    "raw_data.loc[:, 'fare_qcut'] = pd.qcut(raw_data['Fare'], 5)\n",
    "raw_data.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 4.四则运算特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>family_size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "\n",
       "   Parch     Ticket     Fare Cabin Embarked  family_size  \n",
       "0      0  A/5 21171   7.2500   NaN        S            2  \n",
       "1      0   PC 17599  71.2833   C85        C            2  "
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 通过四则运算构建：家庭成员总数 的特征列\n",
    "raw_data.loc[:, 'family_size'] = raw_data['SibSp'] + raw_data['Parch'] + 1\n",
    "raw_data.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>family_size</th>\n",
       "      <th>tmp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>74.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>2</td>\n",
       "      <td>46.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "\n",
       "   Parch     Ticket     Fare Cabin Embarked  family_size   tmp  \n",
       "0      0  A/5 21171   7.2500   NaN        S            2  74.0  \n",
       "1      0   PC 17599  71.2833   C85        C            2  46.0  "
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 年龄 + 客舱等级 + 家族大小 的特征列\n",
    "raw_data.loc[:, 'tmp'] = raw_data['Age'] * raw_data['Pclass'] + 4 * raw_data['family_size']\n",
    "raw_data.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 5.高次项和交叉特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 构造多项式特征\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "# 指定多项式次数：2\n",
    "poly = PolynomialFeatures(degree=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   SibSp  Parch\n",
       "0      1      0\n",
       "1      1      0"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw_data[['SibSp', 'Parch']].head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 1., 0., 1., 0., 0.],\n",
       "       [1., 1., 0., 1., 0., 0.],\n",
       "       [1., 0., 0., 0., 0., 0.],\n",
       "       ...,\n",
       "       [1., 1., 2., 1., 2., 4.],\n",
       "       [1., 0., 0., 0., 0., 0.],\n",
       "       [1., 0., 0., 0., 0., 0.]])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "poly_res = poly.fit_transform(raw_data[['SibSp', 'Parch']])\n",
    "poly_res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 类别型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 1.独热向量编码\n",
    "- pandas get_dummies\n",
    "- OneHotEncoder()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.25</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass                     Name   Sex   Age  SibSp  \\\n",
       "0            1         0       3  Braund, Mr. Owen Harris  male  22.0      1   \n",
       "\n",
       "   Parch     Ticket  Fare Cabin Embarked  \n",
       "0      0  A/5 21171  7.25   NaN        S  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw_data.head(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>C</th>\n",
       "      <th>Q</th>\n",
       "      <th>S</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   C  Q  S\n",
       "0  0  0  1\n",
       "1  1  0  0"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "embarked_hot = pd.get_dummies(raw_data['Embarked'])\n",
    "embarked_hot.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 时间型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 1.时间处理方法：\n",
    "- pandas to_datetime：转为时间格式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date_t</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2012-12-31</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2013-01-02</td>\n",
       "      <td>68.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2013-01-03</td>\n",
       "      <td>36.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2013-01-04</td>\n",
       "      <td>5565.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       date_t     cnt\n",
       "0  2012-12-31     NaN\n",
       "1  2013-01-01     NaN\n",
       "2  2013-01-02    68.0\n",
       "3  2013-01-03    36.0\n",
       "4  2013-01-04  5565.0"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 导入汽车销售数据\n",
    "car_data = pd.read_csv('car_data.csv')\n",
    "car_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1032.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>1760.124031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1153.164214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>12.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1178.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1774.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>2277.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>7226.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               cnt\n",
       "count  1032.000000\n",
       "mean   1760.124031\n",
       "std    1153.164214\n",
       "min      12.000000\n",
       "25%    1178.750000\n",
       "50%    1774.000000\n",
       "75%    2277.750000\n",
       "max    7226.000000"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "car_data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1512 entries, 0 to 1511\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Non-Null Count  Dtype  \n",
      "---  ------  --------------  -----  \n",
      " 0   date_t  1512 non-null   object \n",
      " 1   cnt     1032 non-null   float64\n",
      "dtypes: float64(1), object(1)\n",
      "memory usage: 23.8+ KB\n"
     ]
    }
   ],
   "source": [
    "car_data.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date_t</th>\n",
       "      <th>cnt</th>\n",
       "      <th>date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2012-12-31</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2012-12-31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2013-01-01</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       date_t  cnt       date\n",
       "0  2012-12-31  NaN 2012-12-31\n",
       "1  2013-01-01  NaN 2013-01-01"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 将时间列转为时间格式，并创建新列\n",
    "car_data.loc[:, 'date'] = pd.to_datetime(car_data['date_t'])\n",
    "car_data.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 2.取出关键时间信息\n",
    "- dt.month\n",
    "- dt.dayofweek\n",
    "- dt.dayofyear\n",
    "- ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date_t</th>\n",
       "      <th>cnt</th>\n",
       "      <th>date</th>\n",
       "      <th>month</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2012-12-31</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2012-12-31</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       date_t  cnt       date  month\n",
       "0  2012-12-31  NaN 2012-12-31     12\n",
       "1  2013-01-01  NaN 2013-01-01      1"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 取出月份数据\n",
    "car_data.loc[:, 'month'] = car_data['date'].dt.month\n",
    "car_data.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date_t</th>\n",
       "      <th>cnt</th>\n",
       "      <th>date</th>\n",
       "      <th>month</th>\n",
       "      <th>week</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2012-12-31</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2012-12-31</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       date_t  cnt       date  month  week\n",
       "0  2012-12-31  NaN 2012-12-31     12     0\n",
       "1  2013-01-01  NaN 2013-01-01      1     1"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 取出 星期几 数据\n",
    "car_data.loc[:, 'week'] = car_data['date'].dt.dayofweek\n",
    "car_data.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date_t</th>\n",
       "      <th>cnt</th>\n",
       "      <th>date</th>\n",
       "      <th>month</th>\n",
       "      <th>week</th>\n",
       "      <th>day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2012-12-31</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2012-12-31</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       date_t  cnt       date  month  week  day\n",
       "0  2012-12-31  NaN 2012-12-31     12     0   31\n",
       "1  2013-01-01  NaN 2013-01-01      1     1    1"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 取出是 几号\n",
    "car_data.loc[:, 'day'] = car_data['date'].dt.day\n",
    "car_data.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 文本型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 1.词袋模型\n",
    "- CountVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入词袋模型\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "vectorizer = CountVectorizer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 准备词样本数据\n",
    "text_data = [\n",
    "    'This is a very good class',\n",
    "    'students are very very very good',\n",
    "    'This is the third sentence',\n",
    "    'Is this the last doc'\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<4x12 sparse matrix of type '<class 'numpy.int64'>'\n",
       "\twith 19 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 将文本数据映射为稀疏矩阵\n",
    "text_vector = vectorizer.fit_transform(text_data)\n",
    "text_vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['are',\n",
       " 'class',\n",
       " 'doc',\n",
       " 'good',\n",
       " 'is',\n",
       " 'last',\n",
       " 'sentence',\n",
       " 'students',\n",
       " 'the',\n",
       " 'third',\n",
       " 'this',\n",
       " 'very']"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 打印系数矩阵的列名称\n",
    "vectorizer.get_feature_names()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1],\n",
       "       [1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 3],\n",
       "       [0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0],\n",
       "       [0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0]], dtype=int64)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_vector.toarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 2.N-Gram语言模型\n",
    "> 词袋模型本身是单个词组成的稀疏矩阵，词与词之间无顺序性。词袋模型配合N-Gram语言模型，可以将连接的两个词两两保存下来，保证顺序性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 添加语言模型，指定词种类：单个词、两个词、三个词\n",
    "vec_ngram = CountVectorizer(ngram_range=(1, 3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<4x39 sparse matrix of type '<class 'numpy.int64'>'\n",
       "\twith 48 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 拟合模型\n",
    "text_ngram = vec_ngram.fit_transform(text_data)\n",
    "text_ngram"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['are',\n",
       " 'are very',\n",
       " 'are very very',\n",
       " 'class',\n",
       " 'doc',\n",
       " 'good',\n",
       " 'good class',\n",
       " 'is',\n",
       " 'is the',\n",
       " 'is the third',\n",
       " 'is this',\n",
       " 'is this the',\n",
       " 'is very',\n",
       " 'is very good',\n",
       " 'last',\n",
       " 'last doc',\n",
       " 'sentence',\n",
       " 'students',\n",
       " 'students are',\n",
       " 'students are very',\n",
       " 'the',\n",
       " 'the last',\n",
       " 'the last doc',\n",
       " 'the third',\n",
       " 'the third sentence',\n",
       " 'third',\n",
       " 'third sentence',\n",
       " 'this',\n",
       " 'this is',\n",
       " 'this is the',\n",
       " 'this is very',\n",
       " 'this the',\n",
       " 'this the last',\n",
       " 'very',\n",
       " 'very good',\n",
       " 'very good class',\n",
       " 'very very',\n",
       " 'very very good',\n",
       " 'very very very']"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vec_ngram.get_feature_names()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0],\n",
       "       [1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0,\n",
       "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 0, 2, 1, 1],\n",
       "       [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0,\n",
       "        0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       "       [0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1,\n",
       "        1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0]], dtype=int64)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_ngram.toarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.TF-IDF 特征统计方法\n",
    "- TfidfVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入模型\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "tfidf_vec = TfidfVectorizer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<4x12 sparse matrix of type '<class 'numpy.float64'>'\n",
       "\twith 19 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 特征统计计算\n",
    "text_tfidf = tfidf_vec.fit_transform(text_data)\n",
    "text_tfidf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['are',\n",
       " 'class',\n",
       " 'doc',\n",
       " 'good',\n",
       " 'is',\n",
       " 'last',\n",
       " 'sentence',\n",
       " 'students',\n",
       " 'the',\n",
       " 'third',\n",
       " 'this',\n",
       " 'very']"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tfidf_vec.get_feature_names()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.        , 0.57184829, 0.        , 0.45085176, 0.36500336,\n",
       "        0.        , 0.        , 0.        , 0.        , 0.        ,\n",
       "        0.36500336, 0.45085176],\n",
       "       [0.3488765 , 0.        , 0.        , 0.27505824, 0.        ,\n",
       "        0.        , 0.        , 0.3488765 , 0.        , 0.        ,\n",
       "        0.        , 0.82517473],\n",
       "       [0.        , 0.        , 0.        , 0.        , 0.34432086,\n",
       "        0.        , 0.53944516, 0.        , 0.42530476, 0.53944516,\n",
       "        0.34432086, 0.        ],\n",
       "       [0.        , 0.        , 0.53944516, 0.        , 0.34432086,\n",
       "        0.53944516, 0.        , 0.        , 0.42530476, 0.        ,\n",
       "        0.34432086, 0.        ]])"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_tfidf.toarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 组合特征\n",
    "> 借助条件判断来获取组合特征：判断泰坦尼克号乘客是否是独自一人登船"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>alone</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "\n",
       "   Parch     Ticket     Fare Cabin Embarked  alone  \n",
       "0      0  A/5 21171   7.2500   NaN        S  False  \n",
       "1      0   PC 17599  71.2833   C85        C  False  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raw_data.loc[:, 'alone'] = (raw_data['SibSp'] == 0) & (raw_data['Parch'] == 0)\n",
    "raw_data.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.特征选择"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 过滤式\n",
    "> 评估单个特征和结果值之间的相关程度。但是没有考虑到特征之间的关联作用，可能把有用的关联特征误剔除。以鸢尾花数据为例，进行特征的过滤式选择\n",
    "\n",
    "- SelectKBest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_selection import SelectKBest\n",
    "from sklearn.datasets import load_iris"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on class SelectKBest in module sklearn.feature_selection._univariate_selection:\n",
      "\n",
      "class SelectKBest(_BaseFilter)\n",
      " |  SelectKBest(score_func=<function f_classif at 0x000001F2AE3F8DC0>, *, k=10)\n",
      " |  \n",
      " |  Select features according to the k highest scores.\n",
      " |  \n",
      " |  Read more in the :ref:`User Guide <univariate_feature_selection>`.\n",
      " |  \n",
      " |  Parameters\n",
      " |  ----------\n",
      " |  score_func : callable\n",
      " |      Function taking two arrays X and y, and returning a pair of arrays\n",
      " |      (scores, pvalues) or a single array with scores.\n",
      " |      Default is f_classif (see below \"See also\"). The default function only\n",
      " |      works with classification tasks.\n",
      " |  \n",
      " |      .. versionadded:: 0.18\n",
      " |  \n",
      " |  k : int or \"all\", optional, default=10\n",
      " |      Number of top features to select.\n",
      " |      The \"all\" option bypasses selection, for use in a parameter search.\n",
      " |  \n",
      " |  Attributes\n",
      " |  ----------\n",
      " |  scores_ : array-like of shape (n_features,)\n",
      " |      Scores of features.\n",
      " |  \n",
      " |  pvalues_ : array-like of shape (n_features,)\n",
      " |      p-values of feature scores, None if `score_func` returned only scores.\n",
      " |  \n",
      " |  Examples\n",
      " |  --------\n",
      " |  >>> from sklearn.datasets import load_digits\n",
      " |  >>> from sklearn.feature_selection import SelectKBest, chi2\n",
      " |  >>> X, y = load_digits(return_X_y=True)\n",
      " |  >>> X.shape\n",
      " |  (1797, 64)\n",
      " |  >>> X_new = SelectKBest(chi2, k=20).fit_transform(X, y)\n",
      " |  >>> X_new.shape\n",
      " |  (1797, 20)\n",
      " |  \n",
      " |  Notes\n",
      " |  -----\n",
      " |  Ties between features with equal scores will be broken in an unspecified\n",
      " |  way.\n",
      " |  \n",
      " |  See also\n",
      " |  --------\n",
      " |  f_classif: ANOVA F-value between label/feature for classification tasks.\n",
      " |  mutual_info_classif: Mutual information for a discrete target.\n",
      " |  chi2: Chi-squared stats of non-negative features for classification tasks.\n",
      " |  f_regression: F-value between label/feature for regression tasks.\n",
      " |  mutual_info_regression: Mutual information for a continuous target.\n",
      " |  SelectPercentile: Select features based on percentile of the highest scores.\n",
      " |  SelectFpr: Select features based on a false positive rate test.\n",
      " |  SelectFdr: Select features based on an estimated false discovery rate.\n",
      " |  SelectFwe: Select features based on family-wise error rate.\n",
      " |  GenericUnivariateSelect: Univariate feature selector with configurable mode.\n",
      " |  \n",
      " |  Method resolution order:\n",
      " |      SelectKBest\n",
      " |      _BaseFilter\n",
      " |      sklearn.feature_selection._base.SelectorMixin\n",
      " |      sklearn.base.TransformerMixin\n",
      " |      sklearn.base.BaseEstimator\n",
      " |      builtins.object\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, score_func=<function f_classif at 0x000001F2AE3F8DC0>, *, k=10)\n",
      " |      Initialize self.  See help(type(self)) for accurate signature.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data and other attributes defined here:\n",
      " |  \n",
      " |  __abstractmethods__ = frozenset()\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from _BaseFilter:\n",
      " |  \n",
      " |  fit(self, X, y)\n",
      " |      Run score function on (X, y) and get the appropriate features.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like of shape (n_samples, n_features)\n",
      " |          The training input samples.\n",
      " |      \n",
      " |      y : array-like of shape (n_samples,)\n",
      " |          The target values (class labels in classification, real numbers in\n",
      " |          regression).\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : object\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.feature_selection._base.SelectorMixin:\n",
      " |  \n",
      " |  get_support(self, indices=False)\n",
      " |      Get a mask, or integer index, of the features selected\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      indices : boolean (default False)\n",
      " |          If True, the return value will be an array of integers, rather\n",
      " |          than a boolean mask.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      support : array\n",
      " |          An index that selects the retained features from a feature vector.\n",
      " |          If `indices` is False, this is a boolean array of shape\n",
      " |          [# input features], in which an element is True iff its\n",
      " |          corresponding feature is selected for retention. If `indices` is\n",
      " |          True, this is an integer array of shape [# output features] whose\n",
      " |          values are indices into the input feature vector.\n",
      " |  \n",
      " |  inverse_transform(self, X)\n",
      " |      Reverse the transformation operation\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array of shape [n_samples, n_selected_features]\n",
      " |          The input samples.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      X_r : array of shape [n_samples, n_original_features]\n",
      " |          `X` with columns of zeros inserted where features would have\n",
      " |          been removed by :meth:`transform`.\n",
      " |  \n",
      " |  transform(self, X)\n",
      " |      Reduce X to the selected features.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array of shape [n_samples, n_features]\n",
      " |          The input samples.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      X_r : array of shape [n_samples, n_selected_features]\n",
      " |          The input samples with only the selected features.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.TransformerMixin:\n",
      " |  \n",
      " |  fit_transform(self, X, y=None, **fit_params)\n",
      " |      Fit to data, then transform it.\n",
      " |      \n",
      " |      Fits transformer to X and y with optional parameters fit_params\n",
      " |      and returns a transformed version of X.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : {array-like, sparse matrix, dataframe} of shape                 (n_samples, n_features)\n",
      " |      \n",
      " |      y : ndarray of shape (n_samples,), default=None\n",
      " |          Target values.\n",
      " |      \n",
      " |      **fit_params : dict\n",
      " |          Additional fit parameters.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      X_new : ndarray array of shape (n_samples, n_features_new)\n",
      " |          Transformed array.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from sklearn.base.TransformerMixin:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.BaseEstimator:\n",
      " |  \n",
      " |  __getstate__(self)\n",
      " |  \n",
      " |  __repr__(self, N_CHAR_MAX=700)\n",
      " |      Return repr(self).\n",
      " |  \n",
      " |  __setstate__(self, state)\n",
      " |  \n",
      " |  get_params(self, deep=True)\n",
      " |      Get parameters for this estimator.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      deep : bool, default=True\n",
      " |          If True, will return the parameters for this estimator and\n",
      " |          contained subobjects that are estimators.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      params : mapping of string to any\n",
      " |          Parameter names mapped to their values.\n",
      " |  \n",
      " |  set_params(self, **params)\n",
      " |      Set the parameters of this estimator.\n",
      " |      \n",
      " |      The method works on simple estimators as well as on nested objects\n",
      " |      (such as pipelines). The latter have parameters of the form\n",
      " |      ``<component>__<parameter>`` so that it's possible to update each\n",
      " |      component of a nested object.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      **params : dict\n",
      " |          Estimator parameters.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : object\n",
      " |          Estimator instance.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(SelectKBest)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['DESCR',\n",
       " 'data',\n",
       " 'feature_names',\n",
       " 'filename',\n",
       " 'frame',\n",
       " 'target',\n",
       " 'target_names']"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 载入数据\n",
    "iris_data = load_iris()\n",
    "dir(iris_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150, 4)"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 加载特征值和标签值\n",
    "feature, label = iris_data.data, iris_data.target\n",
    "feature.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150, 3)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_new = SelectKBest(k=3).fit_transform(feature, label)\n",
    "feature_new.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "【结论】：由上可知，特征值数量降低为3。即选择了特征相关度系数 TOP 3 的特征"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 包裹式\n",
    "> 将特征选择看做一个特征子集搜索问题，筛选各种特征子集，用于模型评估效果\n",
    "\n",
    "- RFE：递归特征删除算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 引入递归特征删除算法\n",
    "from sklearn.feature_selection import RFE\n",
    "# 引入随机森林分类器作为训练器使用\n",
    "from sklearn.ensemble import RandomForestClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "rf_model = RandomForestClassifier()\n",
    "# 构建特征递归选择学习器\n",
    "rfe = RFE(rf_model, n_features_to_select=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150, 3)"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 特征递归删除选择\n",
    "feature_rfe = rfe.fit_transform(feature, label)\n",
    "feature_rfe.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5.1, 1.4, 0.2],\n",
       "       [4.9, 1.4, 0.2],\n",
       "       [4.7, 1.3, 0.2],\n",
       "       [4.6, 1.5, 0.2],\n",
       "       [5. , 1.4, 0.2]])"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_rfe[:5, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 嵌入式\n",
    "> 基于 L1 正则化的截断效应，对模型有用的特征进行排序\n",
    "\n",
    "- SelectFromModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 引入 嵌入式 特征选择器\n",
    "from sklearn.feature_selection import SelectFromModel\n",
    "# 引入 线性 SVM\n",
    "from sklearn.svm import LinearSVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on class LinearSVC in module sklearn.svm._classes:\n",
      "\n",
      "class LinearSVC(sklearn.base.BaseEstimator, sklearn.linear_model._base.LinearClassifierMixin, sklearn.linear_model._base.SparseCoefMixin)\n",
      " |  LinearSVC(penalty='l2', loss='squared_hinge', *, dual=True, tol=0.0001, C=1.0, multi_class='ovr', fit_intercept=True, intercept_scaling=1, class_weight=None, verbose=0, random_state=None, max_iter=1000)\n",
      " |  \n",
      " |  Linear Support Vector Classification.\n",
      " |  \n",
      " |  Similar to SVC with parameter kernel='linear', but implemented in terms of\n",
      " |  liblinear rather than libsvm, so it has more flexibility in the choice of\n",
      " |  penalties and loss functions and should scale better to large numbers of\n",
      " |  samples.\n",
      " |  \n",
      " |  This class supports both dense and sparse input and the multiclass support\n",
      " |  is handled according to a one-vs-the-rest scheme.\n",
      " |  \n",
      " |  Read more in the :ref:`User Guide <svm_classification>`.\n",
      " |  \n",
      " |  Parameters\n",
      " |  ----------\n",
      " |  penalty : {'l1', 'l2'}, default='l2'\n",
      " |      Specifies the norm used in the penalization. The 'l2'\n",
      " |      penalty is the standard used in SVC. The 'l1' leads to ``coef_``\n",
      " |      vectors that are sparse.\n",
      " |  \n",
      " |  loss : {'hinge', 'squared_hinge'}, default='squared_hinge'\n",
      " |      Specifies the loss function. 'hinge' is the standard SVM loss\n",
      " |      (used e.g. by the SVC class) while 'squared_hinge' is the\n",
      " |      square of the hinge loss.\n",
      " |  \n",
      " |  dual : bool, default=True\n",
      " |      Select the algorithm to either solve the dual or primal\n",
      " |      optimization problem. Prefer dual=False when n_samples > n_features.\n",
      " |  \n",
      " |  tol : float, default=1e-4\n",
      " |      Tolerance for stopping criteria.\n",
      " |  \n",
      " |  C : float, default=1.0\n",
      " |      Regularization parameter. The strength of the regularization is\n",
      " |      inversely proportional to C. Must be strictly positive.\n",
      " |  \n",
      " |  multi_class : {'ovr', 'crammer_singer'}, default='ovr'\n",
      " |      Determines the multi-class strategy if `y` contains more than\n",
      " |      two classes.\n",
      " |      ``\"ovr\"`` trains n_classes one-vs-rest classifiers, while\n",
      " |      ``\"crammer_singer\"`` optimizes a joint objective over all classes.\n",
      " |      While `crammer_singer` is interesting from a theoretical perspective\n",
      " |      as it is consistent, it is seldom used in practice as it rarely leads\n",
      " |      to better accuracy and is more expensive to compute.\n",
      " |      If ``\"crammer_singer\"`` is chosen, the options loss, penalty and dual\n",
      " |      will be ignored.\n",
      " |  \n",
      " |  fit_intercept : bool, default=True\n",
      " |      Whether to calculate the intercept for this model. If set\n",
      " |      to false, no intercept will be used in calculations\n",
      " |      (i.e. data is expected to be already centered).\n",
      " |  \n",
      " |  intercept_scaling : float, default=1\n",
      " |      When self.fit_intercept is True, instance vector x becomes\n",
      " |      ``[x, self.intercept_scaling]``,\n",
      " |      i.e. a \"synthetic\" feature with constant value equals to\n",
      " |      intercept_scaling is appended to the instance vector.\n",
      " |      The intercept becomes intercept_scaling * synthetic feature weight\n",
      " |      Note! the synthetic feature weight is subject to l1/l2 regularization\n",
      " |      as all other features.\n",
      " |      To lessen the effect of regularization on synthetic feature weight\n",
      " |      (and therefore on the intercept) intercept_scaling has to be increased.\n",
      " |  \n",
      " |  class_weight : dict or 'balanced', default=None\n",
      " |      Set the parameter C of class i to ``class_weight[i]*C`` for\n",
      " |      SVC. If not given, all classes are supposed to have\n",
      " |      weight one.\n",
      " |      The \"balanced\" mode uses the values of y to automatically adjust\n",
      " |      weights inversely proportional to class frequencies in the input data\n",
      " |      as ``n_samples / (n_classes * np.bincount(y))``.\n",
      " |  \n",
      " |  verbose : int, default=0\n",
      " |      Enable verbose output. Note that this setting takes advantage of a\n",
      " |      per-process runtime setting in liblinear that, if enabled, may not work\n",
      " |      properly in a multithreaded context.\n",
      " |  \n",
      " |  random_state : int or RandomState instance, default=None\n",
      " |      Controls the pseudo random number generation for shuffling the data for\n",
      " |      the dual coordinate descent (if ``dual=True``). When ``dual=False`` the\n",
      " |      underlying implementation of :class:`LinearSVC` is not random and\n",
      " |      ``random_state`` has no effect on the results.\n",
      " |      Pass an int for reproducible output across multiple function calls.\n",
      " |      See :term:`Glossary <random_state>`.\n",
      " |  \n",
      " |  max_iter : int, default=1000\n",
      " |      The maximum number of iterations to be run.\n",
      " |  \n",
      " |  Attributes\n",
      " |  ----------\n",
      " |  coef_ : ndarray of shape (1, n_features) if n_classes == 2             else (n_classes, n_features)\n",
      " |      Weights assigned to the features (coefficients in the primal\n",
      " |      problem). This is only available in the case of a linear kernel.\n",
      " |  \n",
      " |      ``coef_`` is a readonly property derived from ``raw_coef_`` that\n",
      " |      follows the internal memory layout of liblinear.\n",
      " |  \n",
      " |  intercept_ : ndarray of shape (1,) if n_classes == 2 else (n_classes,)\n",
      " |      Constants in decision function.\n",
      " |  \n",
      " |  classes_ : ndarray of shape (n_classes,)\n",
      " |      The unique classes labels.\n",
      " |  \n",
      " |  n_iter_ : int\n",
      " |      Maximum number of iterations run across all classes.\n",
      " |  \n",
      " |  See Also\n",
      " |  --------\n",
      " |  SVC\n",
      " |      Implementation of Support Vector Machine classifier using libsvm:\n",
      " |      the kernel can be non-linear but its SMO algorithm does not\n",
      " |      scale to large number of samples as LinearSVC does.\n",
      " |  \n",
      " |      Furthermore SVC multi-class mode is implemented using one\n",
      " |      vs one scheme while LinearSVC uses one vs the rest. It is\n",
      " |      possible to implement one vs the rest with SVC by using the\n",
      " |      :class:`sklearn.multiclass.OneVsRestClassifier` wrapper.\n",
      " |  \n",
      " |      Finally SVC can fit dense data without memory copy if the input\n",
      " |      is C-contiguous. Sparse data will still incur memory copy though.\n",
      " |  \n",
      " |  sklearn.linear_model.SGDClassifier\n",
      " |      SGDClassifier can optimize the same cost function as LinearSVC\n",
      " |      by adjusting the penalty and loss parameters. In addition it requires\n",
      " |      less memory, allows incremental (online) learning, and implements\n",
      " |      various loss functions and regularization regimes.\n",
      " |  \n",
      " |  Notes\n",
      " |  -----\n",
      " |  The underlying C implementation uses a random number generator to\n",
      " |  select features when fitting the model. It is thus not uncommon\n",
      " |  to have slightly different results for the same input data. If\n",
      " |  that happens, try with a smaller ``tol`` parameter.\n",
      " |  \n",
      " |  The underlying implementation, liblinear, uses a sparse internal\n",
      " |  representation for the data that will incur a memory copy.\n",
      " |  \n",
      " |  Predict output may not match that of standalone liblinear in certain\n",
      " |  cases. See :ref:`differences from liblinear <liblinear_differences>`\n",
      " |  in the narrative documentation.\n",
      " |  \n",
      " |  References\n",
      " |  ----------\n",
      " |  `LIBLINEAR: A Library for Large Linear Classification\n",
      " |  <https://www.csie.ntu.edu.tw/~cjlin/liblinear/>`__\n",
      " |  \n",
      " |  Examples\n",
      " |  --------\n",
      " |  >>> from sklearn.svm import LinearSVC\n",
      " |  >>> from sklearn.pipeline import make_pipeline\n",
      " |  >>> from sklearn.preprocessing import StandardScaler\n",
      " |  >>> from sklearn.datasets import make_classification\n",
      " |  >>> X, y = make_classification(n_features=4, random_state=0)\n",
      " |  >>> clf = make_pipeline(StandardScaler(),\n",
      " |  ...                     LinearSVC(random_state=0, tol=1e-5))\n",
      " |  >>> clf.fit(X, y)\n",
      " |  Pipeline(steps=[('standardscaler', StandardScaler()),\n",
      " |                  ('linearsvc', LinearSVC(random_state=0, tol=1e-05))])\n",
      " |  \n",
      " |  >>> print(clf.named_steps['linearsvc'].coef_)\n",
      " |  [[0.141...   0.526... 0.679... 0.493...]]\n",
      " |  \n",
      " |  >>> print(clf.named_steps['linearsvc'].intercept_)\n",
      " |  [0.1693...]\n",
      " |  >>> print(clf.predict([[0, 0, 0, 0]]))\n",
      " |  [1]\n",
      " |  \n",
      " |  Method resolution order:\n",
      " |      LinearSVC\n",
      " |      sklearn.base.BaseEstimator\n",
      " |      sklearn.linear_model._base.LinearClassifierMixin\n",
      " |      sklearn.base.ClassifierMixin\n",
      " |      sklearn.linear_model._base.SparseCoefMixin\n",
      " |      builtins.object\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, penalty='l2', loss='squared_hinge', *, dual=True, tol=0.0001, C=1.0, multi_class='ovr', fit_intercept=True, intercept_scaling=1, class_weight=None, verbose=0, random_state=None, max_iter=1000)\n",
      " |      Initialize self.  See help(type(self)) for accurate signature.\n",
      " |  \n",
      " |  fit(self, X, y, sample_weight=None)\n",
      " |      Fit the model according to the given training data.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : {array-like, sparse matrix} of shape (n_samples, n_features)\n",
      " |          Training vector, where n_samples in the number of samples and\n",
      " |          n_features is the number of features.\n",
      " |      \n",
      " |      y : array-like of shape (n_samples,)\n",
      " |          Target vector relative to X.\n",
      " |      \n",
      " |      sample_weight : array-like of shape (n_samples,), default=None\n",
      " |          Array of weights that are assigned to individual\n",
      " |          samples. If not provided,\n",
      " |          then each sample is given unit weight.\n",
      " |      \n",
      " |          .. versionadded:: 0.18\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : object\n",
      " |          An instance of the estimator.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.BaseEstimator:\n",
      " |  \n",
      " |  __getstate__(self)\n",
      " |  \n",
      " |  __repr__(self, N_CHAR_MAX=700)\n",
      " |      Return repr(self).\n",
      " |  \n",
      " |  __setstate__(self, state)\n",
      " |  \n",
      " |  get_params(self, deep=True)\n",
      " |      Get parameters for this estimator.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      deep : bool, default=True\n",
      " |          If True, will return the parameters for this estimator and\n",
      " |          contained subobjects that are estimators.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      params : mapping of string to any\n",
      " |          Parameter names mapped to their values.\n",
      " |  \n",
      " |  set_params(self, **params)\n",
      " |      Set the parameters of this estimator.\n",
      " |      \n",
      " |      The method works on simple estimators as well as on nested objects\n",
      " |      (such as pipelines). The latter have parameters of the form\n",
      " |      ``<component>__<parameter>`` so that it's possible to update each\n",
      " |      component of a nested object.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      **params : dict\n",
      " |          Estimator parameters.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : object\n",
      " |          Estimator instance.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from sklearn.base.BaseEstimator:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.linear_model._base.LinearClassifierMixin:\n",
      " |  \n",
      " |  decision_function(self, X)\n",
      " |      Predict confidence scores for samples.\n",
      " |      \n",
      " |      The confidence score for a sample is the signed distance of that\n",
      " |      sample to the hyperplane.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array_like or sparse matrix, shape (n_samples, n_features)\n",
      " |          Samples.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      array, shape=(n_samples,) if n_classes == 2 else (n_samples, n_classes)\n",
      " |          Confidence scores per (sample, class) combination. In the binary\n",
      " |          case, confidence score for self.classes_[1] where >0 means this\n",
      " |          class would be predicted.\n",
      " |  \n",
      " |  predict(self, X)\n",
      " |      Predict class labels for samples in X.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array_like or sparse matrix, shape (n_samples, n_features)\n",
      " |          Samples.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      C : array, shape [n_samples]\n",
      " |          Predicted class label per sample.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.ClassifierMixin:\n",
      " |  \n",
      " |  score(self, X, y, sample_weight=None)\n",
      " |      Return the mean accuracy on the given test data and labels.\n",
      " |      \n",
      " |      In multi-label classification, this is the subset accuracy\n",
      " |      which is a harsh metric since you require for each sample that\n",
      " |      each label set be correctly predicted.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like of shape (n_samples, n_features)\n",
      " |          Test samples.\n",
      " |      \n",
      " |      y : array-like of shape (n_samples,) or (n_samples, n_outputs)\n",
      " |          True labels for X.\n",
      " |      \n",
      " |      sample_weight : array-like of shape (n_samples,), default=None\n",
      " |          Sample weights.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      score : float\n",
      " |          Mean accuracy of self.predict(X) wrt. y.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.linear_model._base.SparseCoefMixin:\n",
      " |  \n",
      " |  densify(self)\n",
      " |      Convert coefficient matrix to dense array format.\n",
      " |      \n",
      " |      Converts the ``coef_`` member (back) to a numpy.ndarray. This is the\n",
      " |      default format of ``coef_`` and is required for fitting, so calling\n",
      " |      this method is only required on models that have previously been\n",
      " |      sparsified; otherwise, it is a no-op.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self\n",
      " |          Fitted estimator.\n",
      " |  \n",
      " |  sparsify(self)\n",
      " |      Convert coefficient matrix to sparse format.\n",
      " |      \n",
      " |      Converts the ``coef_`` member to a scipy.sparse matrix, which for\n",
      " |      L1-regularized models can be much more memory- and storage-efficient\n",
      " |      than the usual numpy.ndarray representation.\n",
      " |      \n",
      " |      The ``intercept_`` member is not converted.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self\n",
      " |          Fitted estimator.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      For non-sparse models, i.e. when there are not many zeros in ``coef_``,\n",
      " |      this may actually *increase* memory usage, so use this method with\n",
      " |      care. A rule of thumb is that the number of zero elements, which can\n",
      " |      be computed with ``(coef_ == 0).sum()``, must be more than 50% for this\n",
      " |      to provide significant benefits.\n",
      " |      \n",
      " |      After calling this method, further fitting with the partial_fit\n",
      " |      method (if any) will not work until you call densify.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(LinearSVC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 训练线性支持向量机模型\n",
    "lsvc = LinearSVC(C=0.01, penalty='l1', dual=False, loss='squared_hinge').fit(feature, label)\n",
    "# 构建特征选择器\n",
    "model = SelectFromModel(lsvc, prefit=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150, 3)"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 特征选择\n",
    "feature_select = model.transform(feature)\n",
    "feature_select.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5.1, 3.5, 1.4],\n",
       "       [4.9, 3. , 1.4],\n",
       "       [4.7, 3.2, 1.3],\n",
       "       [4.6, 3.1, 1.5],\n",
       "       [5. , 3.6, 1.4]])"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_select[:5, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
